RMAN Restore stuck up [message #434660] |
Thu, 10 December 2009 02:55 |
chetanaZ
Messages: 132 Registered: October 2009 Location: UK
|
Senior Member |
|
|
Hi,
I am cloning a test database from live backup
In this process i scheduled a script at 03:30 which restores backup from live database backup stored on shared drive
Now the restore started at 03:30 and continued till 05:15 but after which it seems the script is not doing anything. And even the alert.log is not updated after 05:15
I tried to see udump,cdump etc but there isn't any entry
I checked for disk space and there is ample disk space available
Where can i find the issue?
I could see the process by showing command 'ps ef' and it is not appearing in top
Thanks and Regards,
Chetana
Extract of alert.log and sar is as follows
oracle@dev:/opt/oracle/admin/DB01/bdump $tail -100 alert_DB01.log
Thu Dec 10 03:50:01 2009
Full restore complete of datafile 1 to datafile copy /u04/oradata/DB01/system_01.dbf
checkpoint is 5797871699917
Thu Dec 10 03:50:44 2009
Full restore complete of datafile 37 to datafile copy /u04/oradata/DB01/app_index_01.dbf
checkpoint is 5797871699916
Thu Dec 10 03:58:56 2009
Full restore complete of datafile 24 to datafile copy /u04/oradata/DB01/app_data_01.dbf
checkpoint is 5797871699913
Thu Dec 10 03:59:01 2009
Full restore complete of datafile 35 to datafile copy /u04/oradata/DB01/app_migrate_data_04.dbf
checkpoint is 5797871699917
Thu Dec 10 04:00:04 2009
Full restore complete of datafile 32 to datafile copy /u04/oradata/DB01/app_migrate_data_03.dbf
checkpoint is 5797871699916
Thu Dec 10 04:00:14 2009
Full restore complete of datafile 17 to datafile copy /u04/oradata/DB01/app_cwmlite_01.dbf
checkpoint is 5797871699916
Full restore complete of datafile 13 to datafile copy /u04/oradata/DB01/app_crm_data_01.dbf
checkpoint is 5797871699916
Full restore complete of datafile 18 to datafile copy /u04/oradata/DB01/app_drsys_01.dbf
checkpoint is 5797871699916
Thu Dec 10 04:03:06 2009
Full restore complete of datafile 55 to datafile copy /u04/oradata/DB01/app_INDEX.dbf
checkpoint is 5797871699898
Thu Dec 10 04:06:47 2009
Full restore complete of datafile 25 to datafile copy /u04/oradata/DB01/undo03_01.dbf
checkpoint is 5797871699916
Full restore complete of datafile 47 to datafile copy /u04/oradata/DB01/undo03_02.dbf
checkpoint is 5797871699916
Thu Dec 10 04:07:04 2009
Full restore complete of datafile 20 to datafile copy /u04/oradata/DB01/user_01.dbf
checkpoint is 5797871699917
Thu Dec 10 04:07:09 2009
Full restore complete of datafile 26 to datafile copy /u04/oradata/DB01/undo04_01.dbf
checkpoint is 5797871699913
Full restore complete of datafile 48 to datafile copy /u04/oradata/DB01/undo04_02.dbf
checkpoint is 5797871699913
Thu Dec 10 04:07:14 2009
Full restore complete of datafile 19 to datafile copy /u04/oradata/DB01/app_odm_01.dbf
checkpoint is 5797871699917
Thu Dec 10 04:07:29 2009
Full restore complete of datafile 22 to datafile copy /u04/oradata/DB01/undo02_01.dbf
checkpoint is 5797871699898
Full restore complete of datafile 46 to datafile copy /u04/oradata/DB01/undo02_02.dbf
checkpoint is 5797871699898
Thu Dec 10 04:07:39 2009
Full restore complete of datafile 21 to datafile copy /u04/oradata/DB01/xdb_01.dbf
checkpoint is 5797871699913
Thu Dec 10 04:08:13 2009
Full restore complete of datafile 51 to datafile copy /u04/oradata/DB01/driver_data_02.dbf
checkpoint is 5797871699917
Thu Dec 10 04:08:28 2009
Full restore complete of datafile 2 to datafile copy /u04/oradata/DB01/undo01_01.dbf
checkpoint is 5797871699909
Full restore complete of datafile 45 to datafile copy /u04/oradata/DB01/undo01_02.dbf
checkpoint is 5797871699909
Full restore complete of datafile 53 to datafile copy /u04/oradata/DB01/statspack_03.dbf
checkpoint is 5797871699909
Thu Dec 10 04:12:12 2009
Full restore complete of datafile 29 to datafile copy /u04/oradata/DB01/app_cap_index_02.dbf
checkpoint is 5797871699898
Thu Dec 10 04:14:56 2009
Full restore complete of datafile 31 to datafile copy /u04/oradata/DB01/app_migrate_data_02.dbf
checkpoint is 5797871699913
Thu Dec 10 04:15:33 2009
Full restore complete of datafile 14 to datafile copy /u04/oradata/DB01/app_migrate_data_01.dbf
checkpoint is 5797871699913
Thu Dec 10 04:15:40 2009
Full restore complete of datafile 28 to datafile copy /u04/oradata/DB01/app_cap_index_01.dbf
checkpoint is 5797871699898
Thu Dec 10 04:16:01 2009
Full restore complete of datafile 23 to datafile copy /u04/oradata/DB01/statspack_01.dbf
checkpoint is 5797871699898
Thu Dec 10 04:16:24 2009
Full restore complete of datafile 34 to datafile copy /u04/oradata/DB01/system_02.dbf
checkpoint is 5797871699909
Thu Dec 10 04:16:44 2009
Full restore complete of datafile 15 to datafile copy /u04/oradata/DB01/app_migrate_index_01.dbf
checkpoint is 5797871699909
Full restore complete of datafile 30 to datafile copy /u04/oradata/DB01/app_migrate_index_02.dbf
checkpoint is 5797871699909
Thu Dec 10 04:43:26 2009
Full restore complete of datafile 10 to datafile copy /u04/oradata/DB01/app_mr_index_01.dbf
checkpoint is 5797871699917
Thu Dec 10 04:53:22 2009
Full restore complete of datafile 36 to datafile copy /u04/oradata/DB01/app_data_01.dbf
checkpoint is 5797871699917
Thu Dec 10 05:12:23 2009
Full restore complete of datafile 11 to datafile copy /u04/oradata/DB01/driver_index_01.dbf
checkpoint is 5797871699909
Thu Dec 10 05:13:02 2009
Full restore complete of datafile 52 to datafile copy /u04/oradata/DB01/app_las_index_10.dbf
checkpoint is 5797871699917
Thu Dec 10 05:13:48 2009
Full restore complete of datafile 9 to datafile copy /u04/oradata/DB01/app_mr_data_01.dbf
checkpoint is 5797871699909
Thu Dec 10 05:15:21 2009
Full restore complete of datafile 54 to datafile copy /u04/oradata/DB01/app_DATA.dbf
oracle@dev:/opt/oracle/admin/DB01/bdump $sar -b
HP-UX dev B.11.23 U ia64 12/10/09
00:00:00 bread/s lread/s %rcache bwrit/s lwrit/s %wcache pread/s pwrit/s
03:00:01 3722 6549 43 4 10 57 0 0
03:05:01 3826 6853 44 5 12 60 0 0
03:10:01 3204 5713 44 4 11 61 0 0
03:15:01 3606 6404 44 4 9 57 0 0
03:20:01 3707 6628 44 4 10 55 0 0
03:25:01 3646 6482 44 4 18 76 0 0
03:30:01 3211 5678 43 4 10 58 0 0
03:35:01 4759 8238 42 7460 7492 0 0 0
03:40:01 4965 8348 41 9253 9261 0 0 0
03:45:01 4000 6744 41 8567 8577 0 0 0
03:50:01 3292 5746 43 8535 8540 0 0 0
03:55:01 2603 4504 42 9387 9400 0 0 0
04:00:00 2069 3550 42 9777 9780 0 0 0
04:05:00 2131 3655 42 9477 9479 0 0 0
04:10:00 2106 3613 42 9531 9539 0 0 0
04:15:00 2085 3555 41 9561 9567 0 0 0
04:20:00 2099 3691 43 9531 9541 0 0 0
04:25:00 2076 3708 44 9837 9850 0 0 0
04:30:00 2070 4064 49 10060 10063 0 0 0
04:35:00 1970 4042 51 9678 9689 0 0 0
04:40:00 1660 3470 52 8456 8461 0 0 0
04:45:00 1600 3398 53 8387 8391 0 0 0
04:50:00 1623 3465 53 8292 8299 0 0 0
04:55:00 1654 3447 52 8384 8397 0 0 0
05:00:00 1729 3474 50 8450 8453 0 0 0
05:05:00 1688 3502 52 8282 8287 0 0 0
05:10:00 1690 3554 52 8530 8540 0 0 0
05:15:00 1663 3613 54 8760 8766 0 0 0
05:20:00 1027 2374 57 4705 4734 1 0 0
05:25:00 2 741 100 7 24 72 0 0
05:30:00 94 925 90 7 16 59 0 0
05:35:00 252 1331 81 7 17 59 0 0
05:40:00 412 1409 71 6 16 61 0 0
05:45:00 179 1281 86 10 19 50 0 0
05:50:00 672 1540 56 2562 5151 50 0 0
05:55:00 1 752 100 7 26 72 0 0
06:00:00 2 738 100 6 16 60 0 0
06:05:00 0 796 100 7 17 59 0 0
06:10:00 0 742 100 6 16 61 0 0
06:15:00 0 732 100 6 16 60 0 0
06:20:00 0 798 100 7 17 59 0 0
06:25:00 0 749 100 7 24 73 0 0
06:30:00 0 732 100 6 16 61 0 0
06:35:00 0 801 100 7 17 59 0 0
06:40:00 5 748 99 6 16 62 0 0
06:45:00 0 732 100 6 16 60 0 0
06:50:00 0 793 100 7 16 60 0 0
06:55:00 0 760 100 7 25 73 0 0
07:00:00 0 730 100 6 16 59 0 0
07:05:00 0 796 100 7 17 60 0 0
07:10:00 0 748 100 6 16 61 0 0
07:15:00 0 732 100 6 16 61 0 0
07:20:00 0 792 100 7 16 60 0 0
07:25:00 0 749 100 7 24 72 0 0
07:30:00 0 737 100 7 16 60 0 0
07:35:00 0 796 100 7 17 60 0 0
07:40:00 0 743 100 6 16 60 0 0
07:45:00 0 739 100 6 16 60 0 0
07:50:00 0 793 100 7 16 60 0 0
07:55:00 0 750 100 6 24 73 0 0
08:00:01 0 732 100 7 16 58 0 0
08:05:01 0 801 100 7 18 60 0 0
08:10:01 0 742 100 6 16 61 0 0
08:15:01 0 734 100 6 16 59 0 0
08:20:01 0 812 100 7 16 59 0 0
08:25:01 0 750 100 7 24 72 0 0
08:30:01 0 733 100 6 16 60 0 0
08:35:01 1 824 100 9 22 59 0 0
08:40:01 0 760 100 9 22 60 0 0
08:45:01 0 739 100 6 16 60 0 0
08:50:01 1 1262 100 7 19 61 0 0
Average 1794 3468 48 1965 2117 7 0 0
|
|
|
|
Re: RMAN Restore stuck up [message #434703 is a reply to message #434660] |
Thu, 10 December 2009 05:15 |
chetanaZ
Messages: 132 Registered: October 2009 Location: UK
|
Senior Member |
|
|
Mahesh,
many Thanks for the reply
I was about to send the restore script script meanwhile I got message that the issue is resolved
Issue was as following
The System Admin yestrday added disk space and left 'Snap Shot' (Netapp) enabled
Which doubled the volume of actual data and thus ran out of space though showing free space in bdf
Much of this content I am unable to understand but I can see that the restore has resumed juts now
If I am logging as oracle user (not as root), where I can i see such problem where RMAN restore is stuck?
I checked disk space, checked memory, checked script is executing, checked alert.log but could not locate this issue
Is there any file, command where I could have checked this?
My apologies for not sending the script (scrpt extract is given below)
Thanks and Regards,
Chetana
Clone.sh (extract)
$ORACLE_HOME/bin/sqlplus "/ as sysdba" <<EOF>>/u05/scripts/logs/clone.log
startup mount pfile=/u05/scripts/initDB01.ora
exit
EOF
$ORACLE_HOME/bin/rman target / @/u05/scripts/clone.rcv <<EOF>>/u05/scripts/logs/clone.log
exit
EOF
$ORACLE_HOME/bin/sqlplus "/ as sysdba" <<EOF>> /u05/scripts/logs/clone.log
ALTER database open RESETLOGS;
shutdown immediate;
exit
EOF
cat /u05/scripts/clone.rcv
run{
set until time "to_date('2009-12-10:05:30:00','YYYY-MM-DD:HH24:MI:SS')";
set newname for datafile 1 to '/u04/oradata/DB01/system_01.dbf';
set newname for datafile 54 to '/u04/oradata/DB01/app_data_01.dbf';
set newname for datafile 55 to '/u04/oradata/DB01/app_data_02.dbf';
...................................
restore database force;
switch datafile all;
recover database;
}
|
|
|
Re: RMAN Restore stuck up [message #434785 is a reply to message #434703] |
Thu, 10 December 2009 12:26 |
|
Mahesh Rajendran
Messages: 10708 Registered: March 2002 Location: oracleDocoVille
|
Senior Member Account Moderator |
|
|
>>Is there any file, command where I could have checked this?
Either from catalog or from views in target database.
V_$RMAN_BACKUP_JOB_DETAILS VIEW
V_$RMAN_BACKUP_SUBJOB_DETAILS VIEW
V_$RMAN_BACKUP_TYPE VIEW
V_$RMAN_CONFIGURATION VIEW
V_$RMAN_OUTPUT VIEW
V_$RMAN_STATUS VIEW
|
|
|