| Oracle FAQ | Your Portal to the Oracle Knowledge Grid | |
 |
 | |||
| Oracle FAQ | Your Portal to the Oracle Knowledge Grid | |
|
| |||
Home -> Community -> Usenet -> c.d.o.server -> Sporadic Oracle data corruptions errors
Hi,
We have a strange problem of sporadic Oracle data block corruption
errors in our DSS database. Let me start with our environment (which is
complicated, but certified by both Oracle and IBM).
Our environment:
· AIX 5.3 (with and without ML01) with cio enabled, JFS2 filesystem.
· Hardware: IBM P5 P590 and P570 (IBM's latest POWER 5 technology).
· Oracle 9.2.0.5, with data compression.
· SVC (SAN Volume Controller) disk (For people who are not familiar
with SVC, SVC can be connected to multiple, heterogeneous disk arrays.
SVC provides any size, virtual disk to the host. Host needs to be
compatible only with the SVC and not all different disk arrays. It also
provides other bells and whistles like flash copy across arrays, resize
disks and mostly importantly a large cache).
· A DS4100 (FastT) disk array is connected to the SVC. We are planning
to connect more disk array in future.
Actual problem:
Oracle reports sporadic data block corruption error. This is error is
not permanent. For example following is a sequence of events
1. 8:05 AM (normal to low IO load on the server): Oracle reads block
5467 from datafile cposdata1998.dbf. Oracle is able to read and process
the block data.
2. 8:10 AM: (heavy IO load on the server): Oracle reads block 5467 from
datafile cposdata1998.dbf. Oracle is able to read the block, but
reports that the block is corrupted.
3. 8:15 AM (normal to low IO load on the server): Oracle reads block
5467 from datafile cposdata1998.dbf. Oracle is able to read the block,
but reports that the block is corrupted.
4. 9:45 AM (normal to low IO load on the server): Oracle reads block
5467 from datafile cposdata1998.dbf. Oracle is able to read and process
the block.
5. No errors are reported, at any time, by the OS, SVC and disk array.
Only Oracle reports errors.
Following are the patters that we noticed.
· Mostly, errors start when there is heavy IO.
· Sometimes it takes up to a day for the errors to disappear and
sometimes it disappears in a matter of seconds.
· If we copy the datafiles to non-SVC disk, we are not getting this
error.
· We are able recreate this error in compressed and non-compressed
Oracle tablespaces.
Is anybody using a similar environment? If so, please let me know if
you getting similar errors or not. We have severity 1 tickets open with
Oracle and IBM. Oracle says that this is a hardware error and IBM says
that they don't see any error messages.
Thanks,
Joseph
Message from Alert log:
 |
 |