Home » SQL & PL/SQL » SQL & PL/SQL » Fetch out of Sequence (Oracle 10g2, Solaris)
Fetch out of Sequence [message #527697] Wed, 19 October 2011 16:51 Go to next message
mjm22
Messages: 54
Registered: January 2010
Location: Singapore
Member
Hi,

I have an app that reads records from a driver table in order to update another (account) table. The idea is that it runs quickly but does not impact other processes on the system. In total I have around 1M records to update.

Originally I wanted to BULK COLLECT and employ a LIMIT of 1000 with a FORALL. The problem with this approach though is that I make two updates. One to the target table but I also need to update the driver table showing that I have processed the record. Therefore I cannot use this option to commit every time I reach the LIMIT by having a COMMIT inside the LOOP.

So instead I I have a FOR LOOP and test the count, if it is 1000 then I want to commit. I thought this syntax would be fine but I get the 'Fetch Out of Sequence' error. Below is a copy of the code.. Is this being caused by the double update... as the format of the code looks correct to me (though it is late!!)?

PROCEDURE update_set IS

  CURSOR cur_get_recs IS
   SELECT account_num,
          ttw_active_flag,
          acct_rowid,
          rowid driver_rowid
     FROM driver_table
      FOR UPDATE OF processed;

BEGIN
  -- we have multiple DML statements so cannot use FORALL, instead use FOR LOOP
  FOR driver_record IN cur_get_recs LOOP

    BEGIN
      UPDATE account a
         SET ttw_active_flag = driver_record.ttw_active_flag
       WHERE a.rowid = driver_record.acct_rowid;

      -- now update the driver table
      UPDATE driver_table
         SET processed = 'Y'
       WHERE ROWID = driver_record.driver_rowid;

      IF MOD(cur_get_recs%ROWCOUNT,1000) = 0
      THEN
        COMMIT;
      END IF;

    EXCEPTION
      WHEN OTHERS THEN
         dbms_output.put_line('Update failed for Acct: ' || driver_record.account_num ||
                              ' with Error: ' || SQLCODE || ' (' || SQLERRM || ')');
    END;
  END LOOP;
  COMMIT;
END;
Re: Fetch out of Sequence [message #527698 is a reply to message #527697] Wed, 19 October 2011 16:55 Go to previous messageGo to next message
BlackSwan
Messages: 26766
Registered: January 2009
Location: SoCal
Senior Member
>In total I have around 1M records to update.
It will be faster when done as single plain SQL UPDATE & NO PL/SQL.
Re: Fetch out of Sequence [message #527699 is a reply to message #527698] Wed, 19 October 2011 17:13 Go to previous messageGo to next message
mjm22
Messages: 54
Registered: January 2010
Location: Singapore
Member
Hi BlackSwan,

Not an option. I'm tied into the design as the same driver table is used for other processes and I have to update both the target and driver tables for each record processed.

Actually I have realised the issue.. it is because I have declared the CURSOR with a FOR UPDATE OF in order to update the DRIVER table. The Commit is obviously destroying the transaction that this cursor is part of. Once removed the FOR UPDATE the process works fine.

M.
Re: Fetch out of Sequence [message #527702 is a reply to message #527699] Wed, 19 October 2011 17:32 Go to previous messageGo to next message
BlackSwan
Messages: 26766
Registered: January 2009
Location: SoCal
Senior Member
>Not an option. I'm tied into the design as the same driver table is used for other processes and
>I have to update both the target and driver tables for each record processed.

Ever hear of this new fangled thingy called a "transaction".
If COMMIT is issued only after the second table complete single UPDATE, then all stays consistent.
Re: Fetch out of Sequence [message #528350 is a reply to message #527702] Mon, 24 October 2011 07:19 Go to previous messageGo to next message
Kevin Meade
Messages: 2103
Registered: December 1999
Location: Connecticut USA
Senior Member
What Black Swan is saying is that if there are millions of rows in your account table, then bulk updating can be vastly more performant. He is also pointing out that unless you don't have access to the code, your excuse for not doing some kind of bulk update (if you have millions of rows in the acocunt table (which I doubt)), is lame (not intended to be a hurtful comment, we just don't accept lame here).

Consider something like this (I have not checked the syntax of any of this, I leave it to you):

alter table driver_table add update_timestamp timestamp;

declare
   timestamp_v timestamp := sys_timestamp;
begin
   update driver_table set
           update_timestamp = timestamp_v
         , updated = 'Y'
   ;

   update account a set
           ttw_active_flag = (
                               select b.ttw_active_flag
                               from driver_table b
                               where a.rowid = b.acct_rowid
                               and update_timestamp = timestamp_v
                             )
   where rowid in (
                    select acct_rowid
                    from driver_table
                    where update_timestamp = timestamp_v
                  )
   ;

   commit;
end;
/


With this code I have tried to:

1) address the multi-user environment. I assume that multiple processes could be inserting/updating/delete from the driver_table all at the same time. Thus we need some way to lock a set of rows or otherwise guarantee a consist set to work with across multiple statements. You did this with a cursor but cursors are inherently slower as they are all about SLOW BY SLOW processing. I have addressed it by adding a timestamp column and then using that timestamp as a means of finding the same rows later. A sequence number taken from a sequence number generator would work just as well. Some might even say a sequence number would be the only fully correct solution which I suppose it true in a RAC or distributed environment.

2) use regular SQL to do the updates so that the code is not processing one row at a time.

If for some reason you cannot add columns to this table, then you should at the very least look into PLSQL BULK UPDATE. You can keep the same PLSQL you have now, just adapted for array processing.

But if you are only processing a few thousand rows per day or a few hundred, then nobody cares I guess and what you have is fine.

Good luck, Kevin
Re: Fetch out of Sequence [message #528369 is a reply to message #528350] Mon, 24 October 2011 09:03 Go to previous message
mjm22
Messages: 54
Registered: January 2010
Location: Singapore
Member
Hi Both,

Thanks for the input. Perhaps I did not make it clear the reason I cannot write it as a single update as demonstrated above (a more performant option, I agree).....

first we must bear in mind that the account table is a big table and even a straight update (which I have used to rollback the code during testing) takes around 15-30 minutes on a performance test base.

The issue is that there are lots on fire and forget updates coming in from online transactions which update the account table. These online updates are handled through a queuing system. If an update fails, it gets moved to the next queue... where (if my single update is still running) it will fail again. Each time this happens the queue will get halted with the result that there is one less queue processing online transactions. Unfortunately I have no control over this.

This is therefore the reason I require a commit every x number of updates.. When I sent my original post I did indicate I wanted to employ BULK processing, however at this stage I had come to the conclusion that the reason for my 'Fetch out of Sequence' was as a result of the the my bulk processing.

Once I realised that it was the FOR UPDATE clause that was causing my problems I was able to return to the Bulk processing option (or at least in part) and I implemented the following design:

This option is designed to keep the process speedy but at the same time make it more 'friendly' with other processes hitting the account table.

PROCEDURE update_set (p_commit_rate IN NUMBER)
IS

  CURSOR cur_get_recs IS
   SELECT account_num,
          ttw_active_flag,
          acct_rowid,
          rowid driver_rowid
     FROM driver_table;

  TYPE driver_rec IS RECORD (
    account_no      driver_table.account_no%TYPE,
    ttw_active_flag driver_table.ttw_active_flag%TYPE,
    acct_rowid      driver_table.acct_rowid%TYPE,
    driver_rowid    driver_table.drv_rowid%TYPE );

  TYPE driver_recs_tt IS TABLE OF driver_rec;
    
  driver_record  driver_recs_tt;    

BEGIN
  
  OPEN cur_get_recs;
  LOOP
    FETCH cur_get_recs BULK COLLECT INTO driver_record LIMIT p_commit_rate;
    EXIT WHEN driver_record.COUNT = 0;

    -- we have multiple DML statements so cannot use FORALL, instead use FOR LOOP
    FOR i IN 1..driver_record.COUNT
    LOOP  
 
      BEGIN
        UPDATE account a
           SET ttw_active_flag = driver_record(i).ttw_active_flag
         WHERE a.rowid = driver_record(i).acct_rowid;
  
        -- now update the driver table
        UPDATE driver_table
           SET processed = 'Y'
         WHERE ROWID = driver_record(i).driver_rowid;

      EXCEPTION
        -- if either update fails then we get here. Result is no update to driver
        -- table and record will be processed the next time.
      WHEN OTHERS THEN
         dbms_output.put_line('Update failed for Acct: ' || driver_record(i).account_num ||
                              ' with Error: ' || SQLCODE || ' (' || SQLERRM || ')');
      END;
    END LOOP;

    dbms_output.put_line('Commiting ' || driver_record.COUNT);
    -- commit the batch
    COMMIT;

  END LOOP;  

  CLOSE cur_get_driver_recs;

END;

Previous Topic: using cursor to update multiple columns
Next Topic: Need help in identifying parent ID (3 threads merged by bb)
Goto Forum:
  


Current Time: Sun May 18 11:46:15 CDT 2025