| Unable to start crs in a 10g RAC [message #329757] |
Thu, 26 June 2008 07:38  |
divroro12
Messages: 105
Registered: March 2008
Location: Laurel, MD USA
|
Senior Member |
|
|
Hi all,
I have a 2 node 10g RAC running on AIX 5.3 with ASM. Recently i noticed the size (originally 33GB) of my /u01 directory growing very fast & reaching 100%. My backups started failing & after some researching found a trace file over 20G in size. I deleted this file & bounced the database. However, when i did a quick check of my database, i found out that the crs was down. For the past two days i've been unable to start the crs. A query of all my RAC resources using both srvctl & crs_stat (shown below) shows that both listener, ASM & database are down on this server. However sqlplus tells me the contrary (shown below).
(oracle@pgoratt1) /home/oracle > srvctl status db -d pegatt
Instance pegatt1 is not running on node pgoratt1
Instance pegatt2 is running on node pgoratt2
(oracle@pgoratt1) /home/oracle >
(oracle@pgoratt1) /u01/crs/oracle/product/10.2.0/bin > crsstat
HA Resource Target State
----------- ------ -----
ora.pegatt.db ONLINE ONLINE on pgoratt2
ora.pegatt.pegatt1.inst ONLINE OFFLINE
ora.pegatt.pegatt2.inst ONLINE ONLINE on pgoratt2
ora.pgoratt1.ASM1.asm ONLINE OFFLINE
ora.pgoratt1.LISTENER_PGORATT1.lsnr ONLINE OFFLINE
ora.pgoratt1.gsd ONLINE OFFLINE
ora.pgoratt1.ons ONLINE OFFLINE
ora.pgoratt1.vip ONLINE ONLINE on pgoratt1
ora.pgoratt2.ASM2.asm ONLINE ONLINE on pgoratt2
ora.pgoratt2.LISTENER_PGORATT2.lsnr ONLINE ONLINE on pgoratt2
ora.pgoratt2.gsd ONLINE ONLINE on pgoratt2
ora.pgoratt2.ons ONLINE ONLINE on pgoratt2
ora.pgoratt2.vip ONLINE ONLINE on pgoratt2
(oracle@pgoratt1) /u01/crs/oracle/product/10.2.0/bin >
(oracle@pgoratt1) /u01/crs/oracle/product/10.2.0/bin > crsctl start resources
Starting resources.
Error while starting resources. Possible cause: CRSD is down.
SQL> select instance_name,status from gv$instance;
INSTANCE_NAME STATUS
---------------- ------------
pegatt1 OPEN
pegatt2 OPEN
SQL> select instance_name,status from gv$instance;
INSTANCE_NAME STATUS
---------------- ------------
+ASM1 STARTED
+ASM2 STARTED
I'm really confused here because the /u01 directory started increasing in size again, for a reason i can't seem to find since there are no large trace files this time around (increased from 47% to 74% in four days)
I'll be very grateful for any positive input to help me resolve this issue.
Thanks,
[Updated on: Thu, 26 June 2008 09:57] by Moderator Report message to a moderator |
|
|
|
|
|
| Re: Unable to start crs in a 10g RAC [message #330177 is a reply to message #329757] |
Fri, 27 June 2008 15:43  |
divroro12
Messages: 105
Registered: March 2008
Location: Laurel, MD USA
|
Senior Member |
|
|
The trace file was from /u01/crs/oracle/product/10.2.0/crs/trace.
(oracle@pgoratt1) /home/oracle > crsctl check crs
CSS appears healthy
Cannot communicate with CRS
EVM appears healthy
Thanks,
divroro12
|
|
|
|
|
|
|
|
|
|
| Re: Unable to start crs in a 10g RAC [message #330563 is a reply to message #329757] |
Mon, 30 June 2008 08:08 |
divroro12
Messages: 105
Registered: March 2008
Location: Laurel, MD USA
|
Senior Member |
|
|
The result of $CRS_HOME/bin/crs_stat is shown above with a formatted output ($CRS_HOME/bin/crsstat).
(oracle@pgoratt1) /home/oracle > crsctl check install -wait 600
Expecting the CRS daemons to be up within 600 seconds.
CSS is active on these nodes.
pgoratt1
pgoratt2
CSS is active on all nodes.
Thanks,
divroro12
|
|
|
|
|
|
] 
Blog
Search
Help
Members
Register
Login
Home
