OraFAQ Forum: Backup & Recovery » Duplicate: read error on file

Home » RDBMS Server » Backup & Recovery » Duplicate: read error on file (9i, Linux sles8)

Show: Today's Messages :: Polls :: Message Navigator
E-mail to friend

Duplicate: read error on file [message #410634]

Mon, 29 June 2009 07:02

dirkm
Messages: 86
Registered: November 2008
Location: Centurion - South Africa

Member

I am trying to run a duplicate database, from a production server to a test server (same versions).

The backuppieces are mounted (NFS mount), and the permissions are all good (oracle:dba). The duplicate actually starts, but a few datafiles down the line, it suddenly crashes with:

executing command: SET NEWNAME

executing command: SET NEWNAME

executing command: SET NEWNAME

Starting restore at 29-JUN-09

using channel ORA_AUX_DISK_1
using channel ORA_AUX_DISK_2
using channel ORA_AUX_DISK_3
channel ORA_AUX_DISK_1: starting datafile backupset restore
channel ORA_AUX_DISK_1: specifying datafile(s) to restore from backup set
restoring datafile 00017 to /oraindex01/OLTP_dup/mbrchnglogidx001.dbf
restoring datafile 00027 to /oradata01/OLTP_dup/rxtrand003.dbf
restoring datafile 00051 to /oraindex01/OLTP_dup/rxtrndidx002.dbf
restoring datafile 00065 to /oradata01/OLTP_dup/accum006.dbf
channel ORA_AUX_DISK_2: starting datafile backupset restore
channel ORA_AUX_DISK_2: specifying datafile(s) to restore from backup set
restoring datafile 00023 to /oraindex01/OLTP_dup/mbrhistidx001.dbf
restoring datafile 00028 to /oraindex01/OLTP_dup/rxtrndidx001.dbf
restoring datafile 00049 to /oradata01/OLTP_dup/rxtrnlog003.dbf
restoring datafile 00067 to /oradata01/OLTP_dup/rxtrand004.dbf
channel ORA_AUX_DISK_3: starting datafile backupset restore
channel ORA_AUX_DISK_3: specifying datafile(s) to restore from backup set
restoring datafile 00014 to /oraindex01/OLTP_dup/mbr001.dbf
restoring datafile 00031 to /oraindex01/OLTP_dup/rxtrnlogidx001.dbf
restoring datafile 00058 to /oradata01/OLTP_dup/utility01.dbf
restoring datafile 00062 to /oradata01/OLTP_dup/rxtrnlog004.dbf
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03002: failure of Duplicate Db command at 06/29/2009 12:53:25
RMAN-03015: error occurred in stored script Memory Script
ORA-19501: read error on file "/Backup/Rman/prodN1/DBF_PROD_1_35633.RMan", blockno 385 (blocksize=8192)
ORA-27072: skgfdisp: I/O error
Linux Error: 13: Permission denied
Additional information: 384

The permissions on the above file is exactly the same as all the other backuppieces.

Any ideas why I could get this ? Is the backuppiece corrupt - or, could this be network related perhaps ?

Dirk

Report message to a moderator

Re: Duplicate: read error on file [message #410637 is a reply to message #410634]

Mon, 29 June 2009 07:31

ebrian
Messages: 2794
Registered: April 2006

Senior Member

You should verify oracle has read access on the NFS mount by creating a simple text file in /Backup/Rman/prodN1 from the primary and simply try to read the file from the standby.

[Updated on: Mon, 29 June 2009 07:31]

Report message to a moderator

Re: Duplicate: read error on file [message #410680 is a reply to message #410634]

Mon, 29 June 2009 11:26

dirkm
Messages: 86
Registered: November 2008
Location: Centurion - South Africa

Member

Thank you.

I created a testfile in /Backup/Rman/prodN1, under user oracle on the primary (target), and was then able to open the file in VI on the standby (auxiliary). This works.

The file is read-only, but I was able to open it.

Report message to a moderator

Re: Duplicate: read error on file [message #410683 is a reply to message #410680]

Mon, 29 June 2009 11:49

ebrian
Messages: 2794
Registered: April 2006

Senior Member

You may want to set the following mount options for this mount point:

hard,bg,proto=tcp,suid,rsize=32768,wsize=32768,noac

Report message to a moderator

Re: Duplicate: read error on file [message #410804 is a reply to message #410634]

Tue, 30 June 2009 06:12

dirkm
Messages: 86
Registered: November 2008
Location: Centurion - South Africa

Member

Thank you very much. I am running a new backup tonight, on the target, and will then try this again tomorrow morning.

Greatly appreciated
Dirk

Report message to a moderator

Re: Duplicate: read error on file [message #410945 is a reply to message #410683]

Wed, 01 July 2009 03:49

dirkm
Messages: 86
Registered: November 2008
Location: Centurion - South Africa

Member

Thank you ebrian. I just tried your mount options (remounted with the options you suggested).

The duplicate started, but them failed with the same error again, except that the last line keeps changing - the additional information *number*. I don't know why this number would change.

The rest of the error still stays the same, except that I am now restoring (duplicating) from last night's backup (a new backup set).

RMAN-03002: failure of Duplicate Db command at 07/01/2009 10:42:15
RMAN-03015: error occurred in stored script Memory Script
ORA-19501: read error on file "/Backup/Rman/prodN1/DBF_PROD_1_35684.RMan", blockno 5377 (blocksize=8192)
ORA-27072: skgfdisp: I/O error
Linux Error: 2: No such file or directory
Additional information: 5376

Report message to a moderator

Re: Duplicate: read error on file [message #410951 is a reply to message #410634]

Wed, 01 July 2009 04:21

dirkm
Messages: 86
Registered: November 2008
Location: Centurion - South Africa

Member

I am now running the following command, just to test the permissions, and that is working fine - monitoring the output file, I can see that it is being copied & is growing in size:

-->dd if=/Backup/Rman/prodN1/DBF_PROD_1_35684.RMan of=/oradata01/DBF_dd_test.dbf

And I am running this on the duplicate host as user oracle
(reading from the target, writing to the duplicate)

Report message to a moderator

Re: Duplicate: read error on file [message #410954 is a reply to message #410634]

Wed, 01 July 2009 04:30

dirkm
Messages: 86
Registered: November 2008
Location: Centurion - South Africa

Member

The part I don't understand is, the duplicate actually starts creating datafiles:

Data datafiles:
propbmdr01:~ # ls -la /oradata01/OLTP_dup
total 165376
drwxr-xr-x 2 oracle oinstall 4096 Jul 1 11:26 .
drwxr-xr-x 6 oracle oinstall 4096 Jul 1 11:13 ..
-rw-r----- 1 oracle oinstall 6442459136 Jul 1 11:28 accum006.dbf
-rw-r----- 1 oracle oinstall 8589942784 Jul 1 11:28 rxtrand003.dbf
-rw-r----- 1 oracle oinstall 8589942784 Jul 1 11:28 rxtrand004.dbf
-rw-r----- 1 oracle oinstall 9663684608 Jul 1 11:28 rxtrnlog003.dbf
-rw-r----- 1 oracle oinstall 8589942784 Jul 1 11:28 rxtrnlog004.dbf
-rw-r----- 1 oracle oinstall 10737426432 Jul 1 11:28 utility01.dbf

Index datafiles
propbmdr01:~ # ls -la /oraindex01/OLTP_dup
total 175368
drwxr-xr-x 2 oracle oinstall 4096 Jul 1 11:26 .
drwxr-xr-x 6 oracle oinstall 4096 Jun 21 14:05 ..
-rw-r----- 1 oracle oinstall 4294975488 Jul 1 11:28 mbr001.dbf
-rw-r----- 1 oracle oinstall 3221233664 Jul 1 11:28 mbrchnglogidx001.dbf
-rw-r----- 1 oracle oinstall 4294975488 Jul 1 11:28 mbrhistidx001.dbf
-rw-r----- 1 oracle oinstall 5368717312 Jul 1 11:28 rxtrndidx001.dbf
-rw-r----- 1 oracle oinstall 5368717312 Jul 1 11:28 rxtrndidx002.dbf
-rw-r----- 1 oracle oinstall 6442459136 Jul 1 11:28 rxtrnlogidx001.dbf

Then, after a while, it crashes with the error above.

And the permissions on all the backuppieces are the same:

-rw-r----- 1 oracle dba 20902125568 Jun 30 20:00 /Backup/Rman/prodN1/DBF_PROD_1_35682.RMan
-rw-r----- 1 oracle dba 22417244160 Jun 30 20:00 /Backup/Rman/prodN1/DBF_PROD_1_35683.RMan
-rw-r----- 1 oracle dba 23086727168 Jun 30 20:00 /Backup/Rman/prodN1/DBF_PROD_1_35684.RMan
-rw-r----- 1 oracle dba 13380403200 Jun 30 21:21 /Backup/Rman/prodN1/DBF_PROD_1_35685.RMan
-rw-r----- 1 oracle dba 15300132864 Jun 30 21:24 /Backup/Rman/prodN1/DBF_PROD_1_35686.RMan
-rw-r----- 1 oracle dba 12188712960 Jun 30 21:25 /Backup/Rman/prodN1/DBF_PROD_1_35687.RMan
-rw-r----- 1 oracle dba 10672660480 Jun 30 22:05 /Backup/Rman/prodN1/DBF_PROD_1_35688.RMan
-rw-r----- 1 oracle dba 16607854592 Jun 30 22:06 /Backup/Rman/prodN1/DBF_PROD_1_35689.RMan
-rw-r----- 1 oracle dba 7759110144 Jun 30 22:13 /Backup/Rman/prodN1/DBF_PROD_1_35690.RMan
-rw-r----- 1 oracle dba 12319965184 Jun 30 22:42 /Backup/Rman/prodN1/DBF_PROD_1_35691.RMan
-rw-r----- 1 oracle dba 7496228864 Jun 30 22:47 /Backup/Rman/prodN1/DBF_PROD_1_35692.RMan
-rw-r----- 1 oracle dba 13649608704 Jun 30 23:06 /Backup/Rman/prodN1/DBF_PROD_1_35693.RMan
-rw-r----- 1 oracle dba 13526523904 Jun 30 23:13 /Backup/Rman/prodN1/DBF_PROD_1_35694.RMan
-rw-r----- 1 oracle dba 11579105280 Jun 30 23:19 /Backup/Rman/prodN1/DBF_PROD_1_35695.RMan
-rw-r----- 1 oracle dba 12442124288 Jun 30 23:30 /Backup/Rman/prodN1/DBF_PROD_1_35696.RMan
-rw-r----- 1 oracle dba 3878281216 Jun 30 23:33 /Backup/Rman/prodN1/DBF_PROD_1_35697.RMan
-rw-r----- 1 oracle dba 1605124096 Jun 30 23:39 /Backup/Rman/prodN1/DBF_PROD_1_35698.RMan
-rw-r----- 1 oracle dba 800907264 Jun 30 23:42 /Backup/Rman/prodN1/DBF_PROD_1_35699.RMan

Report message to a moderator

Re: Duplicate: read error on file [message #410961 is a reply to message #410954]

Wed, 01 July 2009 05:29

ebrian
Messages: 2794
Registered: April 2006

Senior Member

Oracle, RMAN and NFS inherently have issues, however have you verified the kernel parameter settings and the integrity of the network ? Does /var/log/messages indicate any errors ?

Report message to a moderator

Re: Duplicate: read error on file [message #411177 is a reply to message #410634]

Thu, 02 July 2009 05:05

dirkm
Messages: 86
Registered: November 2008
Location: Centurion - South Africa

Member

I did not find anything in /var/log/messages (also looked there for my previous errors). On the network I checked for receive & transmit errors, but also did not find anything there.

I am now going to try something else. I have installed a fibre card (HBA) on the Test server, and will connect the server to the SAN where the production backups are sitting.

That way I should be able to mount the backup as a local filesystem, and run the restore.

Report message to a moderator

Previous Topic:	problem running RMAN
Next Topic:	RMAN-04005 error

Goto Forum:

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

]

Current Time: Sun Nov 24 23:35:34 CST 2024