Oracle 9i DB Closes Randomly [message #360147] |
Wed, 19 November 2008 15:55 |
mainfr4me
Messages: 5 Registered: November 2008 Location: Waukesha, WI
|
Junior Member |
|
|
Hello,
I've been doing some searching for a while, and have yet to find a good reasoning behind the behavior I'm seeing (and repeated now 2x).
I have a database running on a Windows 2003 Server R2 VMware system, running Oracle 9i (9.2.0.1) version database.
I have a number of these systems out at other locations and customers without hassle, however, I have this one which I cannot keep a system running properly for more than a month or two. I have not seen any new Group Policy items propagate down to the server that would affect it, the site has not installed any additional software that would cause problems, however after a random amount of time and users connected (sometimes happens in the middle of the night, middle of the day, etc), the database goes into a stopped state. I can open the database back up with no errors or trouble, however the database will eventually go back into a closed state.
Has anyone else run into this? I'm not a trained Oracle DB, and I'm doing my best off what I can find on-line (the company decided not to hire another DBA after our last one left), so that's why I've come to ask, HELP?!?!
As a reward, whoever gives me the magical answer will get cookies (and I'm dead serious, I'll mail you cookies this has me tearin my hair out, but I'll make sure none of it makes it into the cookies).
|
|
|
|
Re: Oracle 9i DB Closes Randomly [message #360335 is a reply to message #360154] |
Thu, 20 November 2008 08:08 |
mainfr4me
Messages: 5 Registered: November 2008 Location: Waukesha, WI
|
Junior Member |
|
|
From what I can see, the only entries into the Windows Event Viewer an items stating that the database processes start up, and then nothing other than the normal client connection entries up until the db closes.
I will double check the dump directories and see what I can find in a few minutes.
The systems that these run on are pretty similar, I can't say identical for the Server 2003 systems because more of our sites run on simple XP Pro systems.
Yea, Oracle directly has not been a ball of laughs trying to get help. Although, I have about a dozen of these similar configs running at other sites, so if it would have happened to another site, I would think it would have happened to the others already.
|
|
|
Re: Oracle 9i DB Closes Randomly [message #360359 is a reply to message #360147] |
Thu, 20 November 2008 11:17 |
mainfr4me
Messages: 5 Registered: November 2008 Location: Waukesha, WI
|
Junior Member |
|
|
Alright, good news.
I logged in and found the bdump directory, and I see Saturday when the system initially went offline, there is an entry stating that the system could not write to one of the db files, so it closed to prevent corruption.
What I'm talking to the site now is if they have a backup running or something quirky going on that might have that happen. It reminds me of some of the errors I'd see when a HD would go bad in some of our older systems (not the same error, but similar behavior). I know they use VMotion for high availability, and I'm curious if it's a delayed write to the file and the engine is freaking out over it.
|
|
|
Re: Oracle 9i DB Closes Randomly [message #360362 is a reply to message #360359] |
Thu, 20 November 2008 11:42 |
|
ebrian
Messages: 2794 Registered: April 2006
|
Senior Member |
|
|
Does it coincide with any migration performed with VMotion ? Any errors logged for the disks themselves or disk controller ?
If it appears to be the same file each time, it may not hurt to run dbv against the file in question.
Out of curiosity, has VMotion performed well for your operations ?
[Updated on: Thu, 20 November 2008 11:42] Report message to a moderator
|
|
|
Re: Oracle 9i DB Closes Randomly [message #360363 is a reply to message #360362] |
Thu, 20 November 2008 11:48 |
mainfr4me
Messages: 5 Registered: November 2008 Location: Waukesha, WI
|
Junior Member |
|
|
So far I don't believe any other customers run VMotion, and I've asked the site for more info on the setup and if there were any errors/failovers/etc when this happened on the system.
I had seen something noted that for 10g databases, VMotion seemed to handle it, but during a switch could have a 10-15 second delay (which could account for the write delay on this system).
I have a call with the site's manager in about an hour, so if I find out more then, I'll post it up.
|
|
|
Re: Oracle 9i DB Closes Randomly [message #387526 is a reply to message #360147] |
Thu, 19 February 2009 15:23 |
jrmtl
Messages: 27 Registered: February 2006
|
Junior Member |
|
|
I am having the exact same problem , except i'm not on vmware and this is a physical server.
Have you pinpointed exacty what is going on ?
My database service just stops with no logs anywhere
|
|
|
Re: Oracle 9i DB Closes Randomly [message #387532 is a reply to message #387526] |
Thu, 19 February 2009 15:40 |
mainfr4me
Messages: 5 Registered: November 2008 Location: Waukesha, WI
|
Junior Member |
|
|
Hm - I thought I had updated this.
For this case, it turned out the site admin had decided to run backups both of the VMWare disc (which the VMWare OS handled fine), but also inside the VM's OS backup the individual DB file which would lock the DB files, so when Oracle tried to write, it would detect the file lock and fail.
I had him modify his backup procedure (we were already doing nightly backups via a scheduled task to a zip file on the network) to not backup the OS inside the VM, continue as he wanted to backup the disk files, and all was happy. Unfortunatly, this meant I still had to rebuild that instance again (had corrupted the DB files completely), but we avoided it for the future.
|
|
|