anyone ran into ORA-00752 on dataguard node with ASM/11.1.0.7 Linux?

   We ran into weird corruption in standby nodes in our first production oracle/linux deployment, which is pretty frustrate;

   Would appreciate if anyone else has ran into similar issues and share with your solution; Working with oracle sucks, no progress in the past 2 months and now pointing fingers to HW(hp DL380G7); Which obviously is not the case as all 3 standby always consistently fails at the same block#/log#, and we tried move redo to different location/ASM/filesystem(datafile in primary still at ASM);

   Details:
Application: Oracle RUEI on Linux(redhat 5.5) RDBMS: 11.1.0.7.4
ASM: 11.2.0.2
primary is fine;
dataguard node:

Tue Jul 05 18:10:00 2011

Slave exiting with ORA-752 exception

Errors in file
/oracle/RUEICEM/home/diag/rdbms/rueicem/RUEICEM/trace/RUEICEM_pr06_18594.trc:

*ORA-00752: recovery detected a lost write of a data block*

*ORA-10567: Redo is inconsistent with data block (file# 84, block# 266478)*

ORA-10564: tablespace USERS

ORA-01110: data file 84: '+DATA/rueicem/datafile/users.404.752266499'

ORA-10561: block type 'TRANSACTION MANAGED DATA BLOCK', data object# 5082441

or:

Errors in file
/oracle/RUEICEM/home/products/diag/rdbms/rueicem/RUEICEM/trace/RUEICEM_pr0e_14272.trc (incident=101514):

*ORA-00600: internal error code, arguments: [3020], [84], [266478],
[352588014], [], [], [], [], [], [], [], []*

*ORA-10567: Redo is inconsistent with data block (file# 84, block# 266478)*

ORA-10564: tablespace USERS

ORA-01110: data file 84: '+DATA/rueicem/datafile/rueicem_userruei01_35.dbf'

ORA-10561: block type 'TRANSACTION MANAGED DATA BLOCK', data object# 5082441

Incident details in:
/oracle/RUEICEM/home/products/diag/rdbms/rueicem/RUEICEM/incident/incdir_101514/RUEICEM_pr0e_14272_i101514.trc some of our investigation done:

1. All 3 standby database failed at the same log#/block#; Including the standby with datafile/redo all on filesystem;
2. The standby database cant be recovered even set db_lost_write_protect = none;
3. The standby can still be open after it failed at that block#/log#; DBV is still clean for that datafile by that time; but no way to recover through;
4. The standby can be recovered through through “recover standby database allow 1 corruption”; But dbv did report corrupted block, and fts did report corrupted block and fts fails; Thanks for your sharing, if we dont have any progress, we will have to upgrade RDBMS to 11.2 as well (as 3rd party application, dont really want to do that.)

-- 
Regards
Zhu Chao

--
http://www.freelists.org/webpage/oracle-l