RE: Interesting Issue with RAC - Any Advice Appreciated

From: Bryan Thomas <bthomas_at_perftuning.com>
Date: Thu, 6 Oct 2005 07:22:40 -0500
Message-ID: <EPEOJELOAOKCLPFABGMKIEHEEJAA.bthomas@perftuning.com>

David,

How many times did you reboot the "crashed" server? Sometimes it takes a several tries to get RAC fully back up.

You might also want to try to disable crsd and reboot. Then enable crsd and reboot again. That seems to fix a lot of problems.

I have not worked with RAC on RHEL V4, so I'm not exactly sure what the problem is.

Let me know if any of this helps.

-Bryan Thomas

-----Original Message-----
From: oracle-l-bounce_at_freelists.org
[mailto:oracle-l-bounce_at_freelists.org]On Behalf Of David Barbour
Sent: Wednesday, October 05, 2005 10:37 PM To: Oracle_L
Subject: Interesting Issue with RAC - Any Advice Appreciated

I have an Oracle RAC installed on a pair of Dell PE6850s w/2 processors and 8GB of RAM. Running RHEL 4.0 QU1 and Oracle 10.1.0.4. Using ASM on direct attached CX300 with qLogic HBAs.

Thought everything was just fine - until I was testing the RAC by crashing the nodes. If I crash a node, I see the VIP migrate to the survivor. When I bring up the crashed box, CRSD starts and the VIP migrates back to it's 'home' box, but the other nodeapps, gsd and ons, don't start, and neither does the listener nor the instance. The crsd goes bye-bye (actually 'defunct').

Here's what I see in crs_stat -t:

[oracle_at_rhlv005 ~]$ crs_stat -t

Name Type Target State Host

ora.prod1.db application ONLINE ONLINE rhlv005
ora....11.inst application ONLINE OFFLINE ora....12.inst application ONLINE ONLINE rhlv005
ora....SM1.asm application ONLINE ONLINE rhlv004

ora....04.lsnr application    ONLINE    OFFLINE
ora....004.gsd application    ONLINE    OFFLINE
ora....004.ons application    ONLINE    OFFLINE
ora....004.vip application    ONLINE    ONLINE

rhlv004
ora....SM2.asm application ONLINE ONLINE rhlv005
ora....05.lsnr application ONLINE ONLINE rhlv005
ora....005.gsd application ONLINE ONLINE rhlv005
ora....005.ons application ONLINE ONLINE rhlv005
ora....005.vip application ONLINE ONLINE rhlv005

The crs log is spectacularly informative:

2005-10-05 18:18:34.127: CRS Daemon Started. 2005-10-05 18:18:34.862: Attempting to stop `ora.rhlv004.vip` on member `rhlv005`
2005-10-05 18:18:35.499: Stop of `ora.rhlv004.vip` on member `rhlv005` succeeded.
2005-10-05 18:18:35.720: Attempting to start `ora.rhlv004.vip` on member `rhlv004`
2005-10-05 18:18:41.822: Start of `ora.rhlv004.vip` on member `rhlv004` succeeded.
2005-10-05 18:18:41.922: CRS-1007: Failed after successful dependency consideration

2005-10-05 18:18:33.090: CRSD-1: [CMDMAIN:1336832] Restart waiting for Oracle CRSD to start 2005-10-05 18:18:42.174: CRSD-1: Complete Restart Application Request

Any ideas?

--
http://www.freelists.org/webpage/oracle-l



--
http://www.freelists.org/webpage/oracle-l

Received on Thu Oct 06 2005 - 07:26:25 CDT