Calling SQL LDR and SQL plus scripts in a shell script

Shell Programming and Scripting
1

Hi-
I am trying to achieve the following in a script so I can schedule it on a cron job. I am fairly new to the unix environment...

I have written a shell script that reads a flat file and loads the data into an Oracle table (Table1) via SQLLDR. This Works fine. Then, I run a nested insert statement to parse through the records in Table1 and load the necessary results into a Table. (Table2). This works fine too.

Here is how my script looks like...

# Always truncate table 1 before loading data.
print "truncate table table1;" | sqlplus -s userid/password@db>$now.log
sqlldr userid/password@db control=dbload.ctl>>$now.log
# Always truncate table 2 before loading data.
print "truncate table table2;" | sqlplus -s userid/password@db>>$now.log
# Parse through Table1 and load only necessary records to Table2.
print "insert into table2 select id, min(STARTDATE) as startdate from table1 group by id, to_char(STARTDATE,'mm/dd/yyyy');" | sqlplus -s userid/password@db>>$now.log

I call this script myscript.sh, which will be run via crontab every 15 mins.

Here are my questions:
1) Note how I establish a database connection to execute every sql I need to run? I am thinking there should be an easy way to this. I looked into the "here documents" but not sure how I can use them above. Do I really need to use the "here document" or is my script good the way as it is now?
Would it open and close the db connections for each sql correctly?

If I do use the "here document", do I need to have one for the sqlldr and the other for all the sqlplus part?

Please advise best way to go ahead.
Thanks
RG

2

Using a "here" document would essentially be the same as using a long string. These are equivalent:

print "query1
query2
query3" | sqlplus

cat <<-\EOF | sqlplus
         query1
         query2
         query3
EOF

print "query1" | sqlplus
print "query2" | sqlplus
print "query3" | sqlplus

Either of the first two should be more efficient than the last. Take this example:

# This takes a long time (20-30 sec)
i=0
time while (((i+=1)<100)); do
    rsh localhost date;
done

# This is fast!  1 second!
cat <<-\EOF | rsh localhost sh
                i=0
                while (((i+=1)<100)); do
                        date
                done
                EOF

Since I use "-\EOF", the dash "-" removes any leading tabs from each line. The backslash "\" prevents all the text from being evaluated, and without it I would need to escape any special characters.

3

Thank you. It helps to know that options 1 and 2 are better than 3, which is what I am using now. So, let us say I will go and update my script in line with option 1.

Here are my concerns though...

The following is the order in which my sql commands must run...

1)truncate table1 ==> This is sqlplus command.
2) Load data into table1 ==> This is a sqlldr command.
3) truncate table2 ==> This is sqlplus command.
4) Insert into table2 from table1 ==> This is sqlplus command.

So, #2 is a sqlldr command and not simply a query I can put in sqlplus along with #1, #3 and #4.

How would you recommend that I use options 1 or 2 to achieve the above?
Would I not have to open sqlplus for #1, followed by sqlldr for #2 and then sqlplus again for #3 and #4 combined?

Also, is it okay to have the username and password in the script or should I store it somewhere else? The way I am using it now, can some one use the ps grep command to view the password?

Thank you.
RG

4

Here is what I did...

In the "myscript.sh" script which is my main script, I do the following call...
sqlplus -s @load.sql>$now.log

load.sql is my file that contains all the queries and the sqlldr command
This is what is inside load.sql

username/password@database
truncate table table1;
! sqlldr username/password@database control=dbload.ctl
truncate table table2;
insert into table2 select id, min(STARTDATE) as startdate from table1 group by id, to_char(STARTDATE,'mm/dd/yyyy');
exit

I found out that using the "!" option with in sqlplus gives the control to the OS to run my sqlldr command, complete the dataload and then continue with the remaining 2 sql commands.

Please let me know if I got it all correctly. Should I be looking for anything else?

Thanks
RG

5

You can add error control (before first truncate).
For example :

whenever sqlerror exit failure;
whenever  oserror exit failure;

Jean-Pierre.

6

So here is how my load.sql will look like then?
sorry, I am not much of a sqlplus user either, still learning...
Let us say the below is correct, what would this error control help me achieve?
In other words, what would I achieve by testing for a sqlerror even before I execute my first sql statement? Would my first sql not fail if there was a sql error to begin with?

Please advise.
Thanks
RG

username/password@database
whenever sqlerror exit failure;
whenever oserror exit failure;
truncate table table1;
! sqlldr username/password@database control=dbload.ctl
truncate table table2;
insert into table2 select id, min(STARTDATE) as startdate from table1 group by id, to_char(STARTDATE,'mm/dd/yyyy');
exit