RE: Block corruption again
Date: Wed, 4 Jan 2012 14:17:21 -0500
Message-ID: <E456CBDCBA39124DA45560EA116A22E7C83D39E64A_at_MBLEXCH.mbl.org>
Thanks Jon for the info, this is good to know. It is an Oracle E-Business Suite database on Windows. According to sysadmins, there had been no hardware issue found or reported before or around the time block corruption was occurred. The good news is that they got a failed drive alert after a routine reboot last night. It is one of the drives host the data file which had corrupt blocks. Hope it would be the end of the story after the drive is replaced. Is there any good hardware diagnosis tool that you could recommend? It seems OEM does not help much on this for Windows environment.
-----Original Message-----
From: CRISLER, JON A [mailto:JC1706_at_att.com]
Sent: Wednesday, January 04, 2012 12:55 PM
To: Lu.Jiang_at_umassmed.edu; 'oracle-l_at_freelists.org'
Subject: RE: Block corruption again
By any chance would this be an SAP install? The reason I ask is that I have encountered numerous cases where a fresh install of SAP and Oracle 10g or Oracle 11g encounters corrupted db blocks- we fix them only for them to pop up someplace else. This happens at the tail end of the install !! The basic cause is that there is something goofy in the specific build that SAP ships with its product: I can take the same basic version (say 11.1.0.7), install it on the same server, and never have a problem. Its almost like the seed database provided with SAP has built in corrupted blocks. And we got nowhere with SAP support- they just regurgitate technotes about looking for bad hardware etc.
-----Original Message-----
From: oracle-l-bounce_at_freelists.org [mailto:oracle-l-bounce_at_freelists.org] On Behalf Of Jiang, Lu
Sent: Tuesday, January 03, 2012 4:03 PM
To: 'oracle-l_at_freelists.org'
Subject: RE: Block corruption again
Got this error again. Submitted a SR, hope can get an answer from Oracle.
-----Original Message-----
From: Jiang, Lu
Sent: Tuesday, January 03, 2012 11:03 AM
To: 'JC1706_at_att.com'; 'oracle-l_at_freelists.org'
Subject: RE: Block corruption again
Thanks Jon. Actually I had already given sysadmin a list of Oracle files which should be excluded from anti-virus scan.
-----Original Message-----
From: oracle-l-bounce_at_freelists.org [mailto:oracle-l-bounce_at_freelists.org] On Behalf Of CRISLER, JON A
Sent: Friday, December 30, 2011 1:32 AM
To: Lu.Jiang_at_umassmed.edu; 'oracle-l_at_freelists.org'
Subject: RE: Block corruption again
Generally you want any anti-virus package to not perform any "on-demand" scanning of Oracle disk objects. Besides being a frequent performance bottleneck, it can cause errors as you have encountered. They can also cause some very interesting issues with Oracle RAC.
-----Original Message-----
From: oracle-l-bounce_at_freelists.org [mailto:oracle-l-bounce_at_freelists.org] On Behalf Of Jiang, Lu
Sent: Thursday, December 29, 2011 4:09 PM
To: 'oracle-l_at_freelists.org'
Subject: RE: Block corruption again
Thanks all for your replies.
I had fixed this block corruption last night. Asked sysadmin to make sure no OS level activities against the data files except image backup. Now anti-Virus has been changed to not to scan on data files. I will keep an eye on this and wish the issue won't happen again. Really hope it is not a hardware issue.
Oracle version is 10gR2 with Data Guard, there is no failover/switchover performed recently. The issue seems occurred after rebooting the database machine.
-----Original Message-----
From: CRISLER, JON A [mailto:JC1706_at_att.com]
Sent: Thursday, December 29, 2011 2:58 PM
To: ChrisDavid.Taylor_at_ingrambarge.com; 'howard.latham_at_gmail.com'; 'Jiang, Lu'; 'oracle-l_at_freelists.org'
Subject: RE: Block corruption again
On what Chris is saying- there is a known problem on 11gR1 where objects can get corrupted if you are in a Data Guard environment, and do a switchover and then switchback. Generally it seems to affect indexes, and it will also cause ORA-01555 errors. Rebuilding the index solves the problem until you put in a patch. You should also look into putting in the parameters to enforce a more strict level of block checking.
-----Original Message-----
From: oracle-l-bounce_at_freelists.org [mailto:oracle-l-bounce_at_freelists.org] On Behalf Of Taylor, Chris David
Sent: Thursday, December 29, 2011 8:31 AM
To: 'howard.latham_at_gmail.com'; 'Jiang, Lu'; 'oracle-l_at_freelists.org'
Subject: RE: Block corruption again
Seems to me if this was a hardware problem, and all the datafiles are on the same LUN, then block corruptions would be occurring in more than 1 file.
Being that this is an index datafile, I would assume that it only contains index segments so it might be worthwhile tracking down which index contains the corrupted block.
What version of Oracle are you running, and is there anything special about this database (Dataguard configuration etc etc)?
Chris Taylor
Sr. Oracle DBA
Ingram Barge Company
Nashville, TN 37205
"Quality is never an accident; it is always the result of intelligent effort."
-- John Ruskin (English Writer 1819-1900)
CONFIDENTIALITY NOTICE: This e-mail and any attachments are confidential and may also be privileged. If you are not the named recipient, please notify the sender immediately and delete the contents of this message without disclosing the contents to anyone, using them for any purpose, or storing or copying the information on any medium.
-----Original Message-----
From: oracle-l-bounce_at_freelists.org [mailto:oracle-l-bounce_at_freelists.org] On Behalf Of Howard Latham
Sent: Wednesday, December 28, 2011 1:52 PM
To: Jiang, Lu; oracle-l_at_freelists.org
Subject: RE: Block corruption again
Check and replace hardware
Sent from my Windows Phone
From: Jiang, Lu
Sent: 28/12/2011 19:46
To: oracle-l_at_freelists.org
Subject: Block corruption again
Hi all,
I just got a corrupt block again, had fixed a block corruption in a same index file two weeks ago. Don't know why this happened twice in such a sort period.
I tried rman validate and dbverify to detect the corruption, but rman validate did not report any data corruption while dbverify did.
Could anyone shed any light on this?
Thanks,
Lu
The following is the output from rman validate and dbverify.
Using backup validate:
RMAN> backup check logical validate datafile
'F:\ORACLE\ERPPROD\DB\APPS_ST\DATA\A_TXN_IND01.DBF';
Starting backup at 28-DEC-11
using channel ORA_DISK_1
channel ORA_DISK_1: starting full datafile backupset channel ORA_DISK_1: specifying datafile(s) in backupset input datafile fnochannel ORA_DISK_1: backup set complete, elapsed
time: 00:00:25
Finished backup at 28-DEC-11
Using dbverify:
dbv FILE=F:\ORACLE\ERPPROD\DB\APPS_ST\DATA\A_TXN_IND01.DBF FEEDBACK0 Corrupt block relative dba: 0x06426321 (file 25, block 156449) Bad check value found during dbv:
.......
DBVERIFY - Verification complete
Total Pages Examined : 393216 Total Pages Processed (Data) : 2782 Total Pages Failing (Data) : 0 Total Pages Processed (Index): 93966 Total Pages Failing (Index): 0 Total Pages Processed (Other): 19301 Total Pages Processed (Seg) : 0 Total Pages Failing (Seg) : 0 Total Pages Empty : 277166 Total Pages Marked Corrupt : 1 Total Pages Influx : 0 Highest block SCN : 1326171528 (0.1326171528)
Looks rman validate did not ddetect any data corruption while dbverify did.
--
http://www.freelists.org/webpage/oracle-l
--
http://www.freelists.org/webpage/oracle-l
--
http://www.freelists.org/webpage/oracle-l
--
http://www.freelists.org/webpage/oracle-l
--
http://www.freelists.org/webpage/oracle-l
--
http://www.freelists.org/webpage/oracle-l
--
http://www.freelists.org/webpage/oracle-l
Received on Wed Jan 04 2012 - 13:17:21 CST