creating data loading script

asdfasdfasdfasdf???

What are you asking? asdfasdfasdf is not a question.

Jim,

My question is below ---

We have a below script and based on which we have create a script which starts sqlldr loading.

#!/bin/ksh
# mon_large_files.sh
# Monitors for large files whose BCP out has finished successfully
# and it will then call pgm to run sqlldr for that program
# Calls: DATACITI_orctl_load.sh $FILE_NAME
# where $FILE_NAME is in mon_large_files.dat
# DATACITI_orctl_load.sh must 
#       1. Find correct sqlldr string in DATACITI*_orctl.sh and run it
#       2. If suceessful sends email and comments out entry in DATACITI*_orctl.ch
################################################################################
#export NFS="/DUMPS@uat1"
export NFS="/DUMPS@uat2"
export SCRIPT_DIR="$NFS/dump/scripts"
export LARGEFILE_DAT="$SCRIPT_DIR/mon_large_files.dat"
export LOGDIR="$NFS/sybase_dump/logs"
#LOGFILE name based on FILENAME: strip off dbo piece and dat and add log instead
#Ex: dat=DATACITI_CITI.dbo.trans_data_0000000a_0000089e.dat, log=DATACITI_CITI_trans_data_0000000a_0000089e_bcpOut.log
######################################################################################################################:w
cat $LARGEFILE_DAT | while read DATFILE
do
        #echo DatFile: $DATFILE
        #LOGFILE=`echo $DATFILE| sed s/.dbo./_/g |sed s/".dat"/".log"/g `
        LOGFILE=`echo $DATFILE| nawk '{gsub(".dbo.","_"); $0=substr($0,1,length($0)-4)"_bcpOut.log"; print}'`
        ENDFLAG=`tail $LOGDIR/$LOGFILE| grep -c "Ending bcpz" `
        if [ "$ENDFLAG" -eq 1 ]
        then
                #^If load started already, do not invoke pgm, Check for sqlldr log file or DATACITI_orctl_load.sh does this?
                echo "$LOGFILE indicates completes and load not yet started.."
                echo "\tInvoking [test]: $SCRIPT_DIR/DATACITI_orctl_load.sh $DATFILE"           
        fi
done
#

INFO:
-----
If we run monitor script will call your program(DATACITI_orctl_load.sh): eg: DATACITI_orctl_load.sh $DATFILE

Our program will:

  1. Verify that we have not already loaded this file
#!/bin/bash
lockfile=/tmp/DATACITI_orctl_load.lock
if [ ! -e $lockfile ]; 
then
trap "rm -f $lockfile; 
exit" INT TERM EXIT
touch $lockfile
#call mainprogram
rm $lockfile
trap - INT TERM EXIT
else
echo "DATACITI_orctl_load is already running"
  1. Do the load by finding sqlldr command in DATACITI_*_orctl1n.sh

eg: DATACITI*_orctl1n.sh file contents-
---------------------------------

sqlldr DATACITI_CITI/today05@UAT control=Oracle/dbo_DATACITI_CITI.field_cnt_0000000a_00001d11.ctl direct=true 
columnarrayrows=10000 readsize=104857600 
log=log/dbo_DATACITI_CITI.field_cnt_0000000a_00001d11.log 
bad=log/dbo_DATACITI_CITI.field_cnt_0000000a_00001d11.bad 
discard=log/dbo_DATACITI_CITI.field_cnt_0000000a_00001d11.dis
  1. Once successful, prevent this from being loaded again; (may need to comment out entry in DATACITI_*_orctl1n.sh)
$ cat  mon_large_files.dat
DATACITI_CITI.dbo.trans_data_0000000a_0000089e.dat
DATACITI_CITI.dbo.trans_record.dat
DATACITI_CITI.dbo.field_0000000a_0000089e.dat
DATACITI_CITI.dbo.trans_header.dat
DATACITI_CITI.dbo.field_0000000a_000004b1.dat
DATACITI_CITI.dbo.trans_data_0000000a_00001322.dat
DATACITI_CITI.dbo.trans_data_00000001_00000c09.dat
DATACITI_CITI.dbo.field_00000001_00000c09.dat
DATACITI_CITI.dbo.trans_data_0000000a_000004b1.dat
DATACITI_CITI.dbo.trans_data_0000000a_0000003d.dat

---------- Post updated at 05:00 PM ---------- Previous update was at 04:59 PM ----------

Jim u can change the script in point 1. as well