Read env variables from argument file

Shell Programming and Scripting
1

Hi,
I have a generic shell script (runBatchJob.sh) to read files (batchJob) with commands in them and execute the commands by reading the batchJob file as below

./runBatchJob.sh batchJob

batchJob file

$BATCHDIR/execute_procedure.sh
$DATADIR/collectData.sh
$OTHER_ENV_VAR/doSomething.sh

Now my problem is I'm perfectly able to execute each of the above commands from runBatchJob.sh, $BATCHJOB being replaced by /home/batchDir and so on and so forth.

But when I read the commands out of an argument file in my shell script $BATCHJOB doesn't get replaced at all and I cant run the command. When I echo the statement it prints fine.

Actual code of runBatchJob.sh
$1 being above batchJob file

i=1;
while [ $i -le `wc -l $1 | cut -d" " -f8` ]
    do
        echo Running `head -$i $1 | tail -1`; 
        echo `head -$i $1 | tail -1`;    
        `head -$i $1 | tail -1`;
        i=`expr $i + 1`;
   done;

Please help :stuck_out_tongue:

---------- Post updated at 11:54 AM ---------- Previous update was at 11:39 AM ----------

Ok below worked :slight_smile:
eval `head -$i $1 | tail -1`;

2

I believe in above while loop, you are trying to read each line one by one from batchJob and execute that line. If so, in place of above while loop, you may try:

while read line; do
   echo "Running $line"
   eval $line
done < $1
3

what u need exactly can u explain the question breifly

4

Thanks Anurag but there are additional lines in the file so need to skip them, I actually included only the lines which were needed.

Raja I needed to fetch commands to be executed in a shell script from and input argument file but as the commands being fetched had env variables, they were not being evaluated hence the eval came to the rescue

5

You didn't mention anything about which line to execute and which line to skip. Above while loop will execute all lines in batchJob file.
Pls give some sample of batchJob and some pattern/rule to decide when to execute a line and when not.

6

Ok here is a simple one
first line is name of the batch Job
2nd is the name of the individual job, 3rd is the command for 1st job, 4th is again the name of 2nd individual job, 5th is the 2nd job and ...

myBatchJob
US_SASFeedExtract
$BATCHDIR/execute_procedure.sh prc_generate_sas_bk_feed "(p_success)"
US_SASFeedDataMoveToHist
$BATCHDIR/move.sh "$DATAOUTDIR/GCMSSAS.txt" "$HISTDIR/GCMSSAS.txt$GCMS_DATETIMESTAMP"
US_SASFeedDEODMoveToHist
$BATCHDIR/move.sh "$DATAOUTDIR/GCMSEOD.txt" "$HISTDIR/GCMSEOD.txt$GCMS_DATETIMESTAMP"
US_EmailSASFeed
$BATCHDIR/mail_text.sh $DATAOUTDIR 'GCMSSAS.txt' 'abhijit@xyz.com' "GCMS UAT - SAS Feed File"

Acha in a different note can I have the same batchFeedFile in form of XML and can I read the XML in unix to process it?

7

Why dont u include echo massages in your batch job itself and then simply run that file.

myBatchJob
US_SASFeedExtract
echo "Runnig excute_procedure.sh"
$BATCHDIR/execute_procedure.sh prc_generate_sas_bk_feed "(p_success)"
US_SASFeedDataMoveToHist
echo "Runnig move.sh"
$BATCHDIR/move.sh "$DATAOUTDIR/GCMSSAS.txt" "$HISTDIR/GCMSSAS.txt$GCMS_DATETIMESTAMP"
US_SASFeedDEODMoveToHist
$BATCHDIR/move.sh "$DATAOUTDIR/GCMSEOD.txt" "$HISTDIR/GCMSEOD.txt$GCMS_DATETIMESTAMP"
US_EmailSASFeed
$BATCHDIR/mail_text.sh $DATAOUTDIR 'GCMSSAS.txt' 'abhijit@xyz.com' "GCMS UAT - SAS Feed File"

Than simply execute this file in runBatchJob.sh

..
code
..
sh myBatchJob
..
code
..
8

I understand that you need to execute lines having unix script name i.e. *.sh. Will it hold true across the file?If yes, then good. Following will execute only those lines where 1st column ends with ".sh"

awk '$1 ~ /.*.sh$/' $1 | while read line; do
   echo "Running $line"
   eval $line
done

Yes, You may put batch file in form of XML. Put a proper tag name for command to run and script can read value inside that tag and execute that.

9

Gr8 thanks Anurag, but cud you plz show a small example with the below XML

<batchJob>
<name>myBatchjob</name>
<jobs>
<job>$BATCHDIR/execute_procedure.sh</job>
<job>$BATCHDIR/move.sh</job>
<job>$BATCHDIR/mail_text.sh</job>
</jobs>
</batchJob>
10

Let's say you make your XML job file as:

<batchJob>
<name>myBatchjob</name>
<jobs>
<job>
$BATCHDIR/move.sh "$DATAOUTDIR/GCMSSAS.txt" "$HISTDIR/GCMSSAS.txt$GCMS_DATETIMESTAMP"
</job>
<job>
$BATCHDIR/mail_text.sh $DATAOUTDIR 'GCMSSAS.txt' 'abhijit@xyz.com' "GCMS UAT - SAS Feed File"
</job>
<job>
$BATCHDIR/execute_procedure.sh prc_generate_sas_bk_feed "(p_success)"
</job>
</jobs>
</batchJob>

Then following should do the needful:

awk '/\<job\>/,/\<\/job\>/ if(substr($1,length($1)-2)==".sh") print}' $1 | while read line; do
   echo "Running $line"
   eval $line
done

Here only requirement is:
put your unix script command to be executed inside <job> and </job> as below

<job>
YOUR COMMAND
</job>
11

Ok I'll try above and get back thanks :slight_smile: