Home » RDBMS Server » Server Utilities » SQL Loader and End of File Character
SQL Loader and End of File Character [message #129756] Tue, 26 July 2005 10:04 Go to next message
kuul13
Messages: 8
Registered: March 2005
Junior Member
I have a regular feed coming from my client and a sql loader process in schedule which aacepts the feed and stores in database. Lately we are getting trouble in matching the number of records in feed and loaded into db. SQL Loader is loading one extra row in DB beacuse of an end line character in the feed which the client is sending. We want to handle that scenario in SQLLdr control file. Please help me with this issue. The extract of the feed is something like:

ABC LTD GARGIULO, MARK 2SEMO394W857169
LOP INC DEGROSS, BRIAN 2CLVN144W755207
LOP INC MURAD, ZAHER 3NOCA596U058948
OBTRAILER0520050993398390{0067259962I166460


And control file is:

LOAD DATA
INFILE 'C:\work\clientfeed.txt'
REPLACE
INTO TABLE STG_client
WHEN (1:9) <> 'OBTRAILER'
(
CLIENT position(1:25),
CUST_NAME position(26:50),
CUST_NUM position(51:65)
)

Can anyone let me know how can i solve this without asking client to change the feed.

Thanks
Sri
Re: SQL Loader and End of File Character [message #129757 is a reply to message #129756] Tue, 26 July 2005 10:34 Go to previous messageGo to next message
Mahesh Rajendran
Messages: 10708
Registered: March 2002
Location: oracleDocoVille
Senior Member
Account Moderator
Is this always the last line of the file, that you want to ignore?
Re: SQL Loader and End of File Character [message #129775 is a reply to message #129757] Tue, 26 July 2005 11:55 Go to previous messageGo to next message
kuul13
Messages: 8
Registered: March 2005
Junior Member
Yes this is always like that.
Re: SQL Loader and End of File Character [message #129781 is a reply to message #129775] Tue, 26 July 2005 12:18 Go to previous messageGo to next message
Mahesh Rajendran
Messages: 10708
Registered: March 2002
Location: oracleDocoVille
Senior Member
Account Moderator
Use a little scripting as shown here.
If it is not helping you, let us know.

http://www.orafaq.com/forum/t/27004/0/

Idea is simple:
with load=N parameter you could load only the required lines from datafile.
If number of lines is always the same, that would be simple.
else everytime count the number of lines (N) and loand N-1 lines ( skip the last line).

[Updated on: Tue, 26 July 2005 12:24]

Report message to a moderator

Re: SQL Loader and End of File Character [message #129794 is a reply to message #129781] Tue, 26 July 2005 13:31 Go to previous messageGo to next message
kuul13
Messages: 8
Registered: March 2005
Junior Member
Saw your script and is indeed a good solution but is there any other solution as I am not working on unix. It is a windows system.
Re: SQL Loader and End of File Character [message #129807 is a reply to message #129794] Tue, 26 July 2005 14:56 Go to previous messageGo to next message
Mahesh Rajendran
Messages: 10708
Registered: March 2002
Location: oracleDocoVille
Senior Member
Account Moderator
No problem.
Same principles apply.

C:\sqlldr>rem sample session

C:\sqlldr>type dept.dat
10,ACCOUNTING,NEW YORK
20,RESEARCH,DALLAS
30,SALES,CHICAGO
40,OPERATIONS,BOSTON
trailerrecord
C:\sqlldr>deletelastline.bat dept.dat
        1 file(s) copied.

SQL*Loader: Release 9.2.0.1.0 - Production on Tue Jul 26 15:50:23 2005

Copyright (c) 1982, 2002, Oracle Corporation.  All rights reserved.

Commit point reached - logical record count 4

C:\sqlldr>sqlplus -s scott/tiger@mutation
select * from dept;

    DEPTNO DNAME          LOC
---------- -------------- -------------
        10 ACCOUNTING     NEW YORK
        20 RESEARCH       DALLAS
        30 SALES          CHICAGO
        40 OPERATIONS     BOSTON

exit;


C:\sqlldr>rem codes used
 
C:\sqlldr>type deletelastline.bat
@echo off
rem ----------------------------------------------------------
rem following are comments
rem ----------------------------------------------------------
rem deletelastline.bat is the batch file that takes two inputs
rem first input name of the file whose line(s) need to be deleted
rem        start from lastline
rem second input count of lines to be deleted.
rem        default is 1.
rem Just to be in safe side, before deleting the last i am copying
rem         the original file to filename.moved.
rem Now use sqlldr as usual and load the text file
rem ------------------------------------------------------------
copy %1 %1.moved
if {%1}=={}  @echo Syntax: DelLastLines FileName [NumLines]&goto :EOF
if not exist %1 @echo DelLastLines File %1 not found.&goto :EOF
setlocal ENABLEDELAYEDEXPANSION
set /a num=1
if not {%2}=={} set /a num=%2
set wrk="%TEMP%\DelLastLines.tmp"
set fle=%1
if exist %wrk% del /q %wrk%
for /f %%n in ('find /V /C "" ^<%fle%') do set /a nbr=%%n
set /a nbr=%nbr% - %num%
for /f "tokens=1,* delims=[]" %%K in ('find /V /N "" ^<%fle%') do (
 set /a cnt=%%K
 if !cnt! LEQ %nbr% @echo/%%L>>%wrk%
)
call :cpy >nul 2>&1
del /q %wrk%
endlocal
goto :sqlload
:cpy
copy /Y %wrk% %fle%
:sqlload
rem -------------------------
rem invoke sql*loader
rem i have hardcoded the filename dept.dat in control file.
rem you can pass it in runtime.
rem -------------------------
sqlldr userid=scott/tiger@mutation control=dept.ctl
goto :EOF

C:\sqlldr>type dept.ctl
LOAD DATA
infile 'dept.dat'
replace INTO TABLE dept FIELDS TERMINATED BY "," trailing nullcols
(
deptno,
dname,
loc
)

C:\sqlldr>
Re: SQL Loader and End of File Character [message #129812 is a reply to message #129807] Tue, 26 July 2005 15:20 Go to previous messageGo to next message
kuul13
Messages: 8
Registered: March 2005
Junior Member
Too good a solution. Thanks.
This solution was as per my reply whether the file will always have this end line character, but is there any way to check whether there is an end-line character then do this otherwise do normal processing otherwise it will always delete the last line. Synonymous to WHEN clause.
Re: SQL Loader and End of File Character [message #129814 is a reply to message #129812] Tue, 26 July 2005 15:24 Go to previous message
Mahesh Rajendran
Messages: 10708
Registered: March 2002
Location: oracleDocoVille
Senior Member
Account Moderator
>>but is there any way to check whether there is an end-line character
You have to comeup with a list of possible endline characters ( even if it is in the same line. i.e,endlinecharacter need NOT to be in new line/lastline. We can also do a last N characters in the last line).
I am not too comfortable with windows scripting. can give a try.

[Updated on: Tue, 26 July 2005 15:27]

Report message to a moderator

Previous Topic: I really need your help .please help me (sqlloader poblem)
Next Topic: Problem Loading data using sqlldr
Goto Forum:
  


Current Time: Thu Dec 26 20:37:40 CST 2024