Home » SQL & PL/SQL » SQL & PL/SQL » Fetch out of Sequence (Oracle 10g2, Solaris)
Fetch out of Sequence [message #527697] |
Wed, 19 October 2011 16:51  |
mjm22
Messages: 54 Registered: January 2010 Location: Singapore
|
Member |
|
|
Hi,
I have an app that reads records from a driver table in order to update another (account) table. The idea is that it runs quickly but does not impact other processes on the system. In total I have around 1M records to update.
Originally I wanted to BULK COLLECT and employ a LIMIT of 1000 with a FORALL. The problem with this approach though is that I make two updates. One to the target table but I also need to update the driver table showing that I have processed the record. Therefore I cannot use this option to commit every time I reach the LIMIT by having a COMMIT inside the LOOP.
So instead I I have a FOR LOOP and test the count, if it is 1000 then I want to commit. I thought this syntax would be fine but I get the 'Fetch Out of Sequence' error. Below is a copy of the code.. Is this being caused by the double update... as the format of the code looks correct to me (though it is late!!)?
PROCEDURE update_set IS
CURSOR cur_get_recs IS
SELECT account_num,
ttw_active_flag,
acct_rowid,
rowid driver_rowid
FROM driver_table
FOR UPDATE OF processed;
BEGIN
-- we have multiple DML statements so cannot use FORALL, instead use FOR LOOP
FOR driver_record IN cur_get_recs LOOP
BEGIN
UPDATE account a
SET ttw_active_flag = driver_record.ttw_active_flag
WHERE a.rowid = driver_record.acct_rowid;
-- now update the driver table
UPDATE driver_table
SET processed = 'Y'
WHERE ROWID = driver_record.driver_rowid;
IF MOD(cur_get_recs%ROWCOUNT,1000) = 0
THEN
COMMIT;
END IF;
EXCEPTION
WHEN OTHERS THEN
dbms_output.put_line('Update failed for Acct: ' || driver_record.account_num ||
' with Error: ' || SQLCODE || ' (' || SQLERRM || ')');
END;
END LOOP;
COMMIT;
END;
|
|
|
|
Re: Fetch out of Sequence [message #527699 is a reply to message #527698] |
Wed, 19 October 2011 17:13   |
mjm22
Messages: 54 Registered: January 2010 Location: Singapore
|
Member |
|
|
Hi BlackSwan,
Not an option. I'm tied into the design as the same driver table is used for other processes and I have to update both the target and driver tables for each record processed.
Actually I have realised the issue.. it is because I have declared the CURSOR with a FOR UPDATE OF in order to update the DRIVER table. The Commit is obviously destroying the transaction that this cursor is part of. Once removed the FOR UPDATE the process works fine.
M.
|
|
|
|
Re: Fetch out of Sequence [message #528350 is a reply to message #527702] |
Mon, 24 October 2011 07:19   |
 |
Kevin Meade
Messages: 2103 Registered: December 1999 Location: Connecticut USA
|
Senior Member |
|
|
What Black Swan is saying is that if there are millions of rows in your account table, then bulk updating can be vastly more performant. He is also pointing out that unless you don't have access to the code, your excuse for not doing some kind of bulk update (if you have millions of rows in the acocunt table (which I doubt)), is lame (not intended to be a hurtful comment, we just don't accept lame here).
Consider something like this (I have not checked the syntax of any of this, I leave it to you):
alter table driver_table add update_timestamp timestamp;
declare
timestamp_v timestamp := sys_timestamp;
begin
update driver_table set
update_timestamp = timestamp_v
, updated = 'Y'
;
update account a set
ttw_active_flag = (
select b.ttw_active_flag
from driver_table b
where a.rowid = b.acct_rowid
and update_timestamp = timestamp_v
)
where rowid in (
select acct_rowid
from driver_table
where update_timestamp = timestamp_v
)
;
commit;
end;
/
With this code I have tried to:
1) address the multi-user environment. I assume that multiple processes could be inserting/updating/delete from the driver_table all at the same time. Thus we need some way to lock a set of rows or otherwise guarantee a consist set to work with across multiple statements. You did this with a cursor but cursors are inherently slower as they are all about SLOW BY SLOW processing. I have addressed it by adding a timestamp column and then using that timestamp as a means of finding the same rows later. A sequence number taken from a sequence number generator would work just as well. Some might even say a sequence number would be the only fully correct solution which I suppose it true in a RAC or distributed environment.
2) use regular SQL to do the updates so that the code is not processing one row at a time.
If for some reason you cannot add columns to this table, then you should at the very least look into PLSQL BULK UPDATE. You can keep the same PLSQL you have now, just adapted for array processing.
But if you are only processing a few thousand rows per day or a few hundred, then nobody cares I guess and what you have is fine.
Good luck, Kevin
|
|
|
Re: Fetch out of Sequence [message #528369 is a reply to message #528350] |
Mon, 24 October 2011 09:03  |
mjm22
Messages: 54 Registered: January 2010 Location: Singapore
|
Member |
|
|
Hi Both,
Thanks for the input. Perhaps I did not make it clear the reason I cannot write it as a single update as demonstrated above (a more performant option, I agree).....
first we must bear in mind that the account table is a big table and even a straight update (which I have used to rollback the code during testing) takes around 15-30 minutes on a performance test base.
The issue is that there are lots on fire and forget updates coming in from online transactions which update the account table. These online updates are handled through a queuing system. If an update fails, it gets moved to the next queue... where (if my single update is still running) it will fail again. Each time this happens the queue will get halted with the result that there is one less queue processing online transactions. Unfortunately I have no control over this.
This is therefore the reason I require a commit every x number of updates.. When I sent my original post I did indicate I wanted to employ BULK processing, however at this stage I had come to the conclusion that the reason for my 'Fetch out of Sequence' was as a result of the the my bulk processing.
Once I realised that it was the FOR UPDATE clause that was causing my problems I was able to return to the Bulk processing option (or at least in part) and I implemented the following design:
This option is designed to keep the process speedy but at the same time make it more 'friendly' with other processes hitting the account table.
PROCEDURE update_set (p_commit_rate IN NUMBER)
IS
CURSOR cur_get_recs IS
SELECT account_num,
ttw_active_flag,
acct_rowid,
rowid driver_rowid
FROM driver_table;
TYPE driver_rec IS RECORD (
account_no driver_table.account_no%TYPE,
ttw_active_flag driver_table.ttw_active_flag%TYPE,
acct_rowid driver_table.acct_rowid%TYPE,
driver_rowid driver_table.drv_rowid%TYPE );
TYPE driver_recs_tt IS TABLE OF driver_rec;
driver_record driver_recs_tt;
BEGIN
OPEN cur_get_recs;
LOOP
FETCH cur_get_recs BULK COLLECT INTO driver_record LIMIT p_commit_rate;
EXIT WHEN driver_record.COUNT = 0;
-- we have multiple DML statements so cannot use FORALL, instead use FOR LOOP
FOR i IN 1..driver_record.COUNT
LOOP
BEGIN
UPDATE account a
SET ttw_active_flag = driver_record(i).ttw_active_flag
WHERE a.rowid = driver_record(i).acct_rowid;
-- now update the driver table
UPDATE driver_table
SET processed = 'Y'
WHERE ROWID = driver_record(i).driver_rowid;
EXCEPTION
-- if either update fails then we get here. Result is no update to driver
-- table and record will be processed the next time.
WHEN OTHERS THEN
dbms_output.put_line('Update failed for Acct: ' || driver_record(i).account_num ||
' with Error: ' || SQLCODE || ' (' || SQLERRM || ')');
END;
END LOOP;
dbms_output.put_line('Commiting ' || driver_record.COUNT);
-- commit the batch
COMMIT;
END LOOP;
CLOSE cur_get_driver_recs;
END;
|
|
|
Goto Forum:
Current Time: Sun May 18 11:46:15 CDT 2025
|