Re: Shutdown -halt with RAC

From: Henry Poras <henry_at_itasoftware.com>
Date: Tue, 15 Nov 2005 21:52:13 -0500
Message-ID: <437A9EDD.4070709@itasoftware.com>

Here is the crs log. I guess what I was wondering was when Oracle will reboot a node and when it will shut it down. Documentation seems to be weak when it comes to describing these scenarios.

Thanks.

Henry

2005-11-10 15:17:54.024: CRSD-1: Sending Restart Application Request Oracle Database 10g CRS Release 10.1.0.4.0 Production Copyright 1996, 2004, Oracle. All rights reserved.
2005-11-10 15:17:54 : Oracle CRSD will start boot services 2005-11-10 15:17:54 : Changing directory to /ita/crs/oracle/product/10.1.0/crs/crs/init/ 2005-11-10 15:17:54 : Wrote pid 13595 to /ita/crs/oracle/product/10.1.0/crs/crs/init/.lock-serverA

2005-11-10 15:17:54.053: CRS Daemon Starting
2005-11-10 15:17:54.053: Checking the OCR device
2005-11-10 15:17:54.090: Connecting to the CSS Daemon
2005-11-10 15:17:54.154: Initializing OCR
2005-11-10 15:17:54.245: Writing Software Version to OCR
2005-11-10 15:17:54.321: Using Authorizer location:

/ita/crs/oracle/product/10.1.0/crs/crs/auth/

2005-11-10 15:17:54.379: Initializing RTI
2005-11-10 15:17:54.383: Parameter SECURITY = 1, running in USER Mode
2005-11-10 15:17:54.383: Initializing EVMMgr
2005-11-10 15:18:58.687: CRSD locked during state recovery, please wait.
2005-11-10 15:18:59.188: CRSD recovered, unlocked.
2005-11-10 15:18:59.190: QS socket on:

(ADDRESS=(PROTOCOL=ipc)(KEY=ora_crsqs))
2005-11-10 15:18:59.191: UI socket on:
(ADDRESS=(PROTOCOL=ipc)(KEY=serverA_crs_caa)) 2005-11-10 15:18:59.192: E2E socket on: (ADDRESS=(PROTOCOL=tcp)(HOST=serverAprv)(PORT=49896))

2005-11-10 15:18:59.192: Starting Threads
2005-11-10 15:18:59.192: CRS Daemon Started.
2005-11-10 15:19:00.470: CRS-1007: Failed after successful dependency

consideration

2005-11-10 15:19:01.199: Attempting to start `ora.serverA.vip` on member `serverA`
2005-11-10 15:19:01.783: Start of `ora.serverA.vip` on member `serverA` succeeded.
2005-11-10 15:19:01.893: Attempting to start `ora.serverA.ASM2.asm` on member `serverA`

2005-11-10 15:19:03.183: Start of `ora.serverA.ASM2.asm` on member 
`serverA` succeeded.
2005-11-10 15:19:04.033: Attempting to start

`ora.serverA.LISTENER_serverA.lsnr` on member `serverA` 2005-11-10 15:19:04.619: Start of `ora.serverA.LISTENER_serverA.lsnr` on member `serverA` succeeded.
2005-11-10 15:19:05.045: Attempting to start `ora.serverA.gsd` on member `serverA`
2005-11-10 15:19:05.498: Start of `ora.serverA.gsd` on member `serverA` succeeded.
2005-11-10 15:19:05.905: Attempting to start `ora.serverA.ons` on member `serverA`
2005-11-10 15:19:06.259: Start of `ora.serverA.ons` on member `serverA` succeeded.
2005-11-10 15:19:07.152: Attempting to start `ora.test.test2.inst` on member `serverA`

2005-11-10 15:19:11.412: Start of `ora.test.test2.inst` on member 
`serverA` succeeded.
2005-11-10 15:17:56.158: CRSD-1: [CMDMAIN:3059703936] Restart waiting

for Oracle CRSD to start
2005-11-10 15:18:24.256: CRSD-1: [CMDMAIN:3059703936] Restart waiting for Oracle CRSD to start
2005-11-10 15:18:52.357: CRSD-1: [CMDMAIN:3059703936] Restart waiting for Oracle CRSD to start
2005-11-10 15:19:11.488: CRSD-1: Complete Restart Application Request 2005-11-10 15:29:06.558: [RUNCONTEXT::RUNSCRIPT:2529962928] CheckResource error for ora.serverA.ons error code = 1 `ora.serverA.ons` on `serverA` went OFFLINE unexpectedly 2005-11-10 15:29:06.676: Attempting to stop `ora.serverA.ons` on member `serverA`
2005-11-10 15:29:06.898: Stop of `ora.serverA.ons` on member `serverA` succeeded.
Restarting `ora.serverA.ons` on `serverA` 2005-11-10 15:29:06.962: Attempting to start `ora.serverA.ons` on member `serverA`
2005-11-10 15:29:07.179: Start of `ora.serverA.ons` on member `serverA` succeeded.
Successfully restarted `ora.serverA.ons` on `serverA` 2005-11-10 15:31:05.078: ALERT [CAAOCRLOOKUP:2624658352] OCR api procr_open_key failed for key CRS.CUR.ora!serverA!vip OCR error code = 3 OCR error msg:
crsd.bin: caaocr.cpp:200: Assertion `0==1' failed.

2005-11-10 15:33:41.329: CRSD-2: Sending Shutdown Application Request
PROC-22: The OCR backend has an invalid format
PROC-22: The OCR backend has an invalid format
PROC-22: The OCR backend has an invalid format
PROC-22: The OCR backend has an invalid format
PROC-22: The OCR backend has an invalid format
2005-11-10 15:33:55.533: CRSD-2: ALERT [MAIN:3059703936] Error sending

request to running CRSD
2005-11-10 15:33:55.533: CRSD-2: ALERT [MAIN:3059703936] SHUTDOWNAPPS Could not send application shutdown request to CRSD 2005-11-10 15:38:06.143: CRSD-1: Sending Restart Application Request

K Gopalakrishnan wrote:

>Henry:
>
>What does your CRS log file say? I guess CRS killed instance to avoid
>data corruption. THis seems like a simple io-fencing issue.
>
>Regards,
>Gopal
>
>
>On 11/15/05, Henry Poras <henry_at_itasoftware.com> wrote:
>
>
>>I've been playing around with a RAC system (10.1.0.4 on RedHat AS3), and
>>managed to crash it(no problem, I'm trying to see what can go wrong). It was
>>probably due to a corrupted Registry file (does ocrconfig -import work?). On
>>trying to export and then import a Registry file, then restart CRS, one of
>>our nodes removed itself (no longer visible from the network. Probable
>>crash). When our SysAdm went down to look, he said Oracle had 'halt'ed the
>>server. All the doc I have seen talks about 'reboot' under certain
>>conditions, but I have never seen a halt (or shutdown) mentioned anywhere.
>>Has anyone else seen (or seen documentation of) this behavior?
>>
>>Henry
>>
>>
>
>
>--
>Best Regards,
>K Gopalakrishnan
>Co-Author: Oracle Wait Interface, Oracle Press 2004
>http://www.amazon.com/exec/obidos/tg/detail/-/007222729X/
>
>
>
>

--
http://www.freelists.org/webpage/oracle-l

Received on Tue Nov 15 2005 - 20:55:21 CST