How to WAIT for jobs in each group to finish?

Shell Programming and Scripting
1

I have the shell script to call a Perl routine and pass the Informatica WorkFlow name to it. Jobs in each group executes in background do not seem to wait at all. How do I make it to WAIT for the prior group to complete before execute the next group of jobs?

Sample of the jobs flow:

{
 
 echo "=========>> Executing Group-1 at `date` "
 {
   `./ProcessData.pl wf_LOAD_EDW_Table1 >> ${LogDir}/ProcessDataGroup1.log 2>&1 &`;  
   `./ProcessData.pl wf_LOAD_EDW_Table2 >> ${LogDir}/ProcessDataGroup1.log 2>&1 &`;  
 }
 wait
 echo "=========>> Executing Group-1 Has Completed at `date` "
 echo ""
 echo "=========>> Executing Group-2 at `date` "
 {
   `./ProcessData.pl wf_LOAD_EDW_Table3 >> ${LogDir}/ProcessDataGroup2.log 2>&1 &`;  
   `./ProcessData.pl wf_LOAD_EDW_Table4 >> ${LogDir}/ProcessDataGroup2.log 2>&1 &`;  
 }
 wait
 echo "=========>> Executing Group-2 Has Completed at `date` "
 echo ""
 echo "=========>> Executing Group-3 at `date` "
 { 
   `./ProcessData.pl wf_LOAD_EDW_Table5 >> ${LogDir}/ProcessDataGroup3.log 2>&1 &`;  
   `./ProcessData.pl wf_LOAD_EDW_Table6 >> ${LogDir}/ProcessDataGroup2.log 2>&1 &`;  
 }
 wait
 echo "=========>> Executing Group-3 Has Completed at `date` "

}
2

Is there any reason for which you are putting your commands into backticks (`...`)? Try removing them and run the script again.

3 
echo "=========>> EDW_Daily_Sch_DS_step7 Group-1 Job Started at `date` "
   `sleep 10 &`
   `sleep 30 &`
 wait
 echo "=========>> EDW_Daily_Sch_DS_step7 Group-1 Has Completed at `date` "
 echo ""
 echo "=========>> EDW_Daily_Sch_DS_step7 Group-2 Job Started at `date` "
   `sleep 30 &`
   `sleep 10 &`
 wait
 echo "=========>> EDW_Daily_Sch_DS_step7 Group-2 Has Completed at `date` "

Result:

=========>> EDW_Daily_Sch_DS_step7 Group-1 Job Started at Sat Sep 10 13:47:38 PDT 2011
=========>> EDW_Daily_Sch_DS_step7 Group-1 Has Completed at Sat Sep 10 13:48:18 PDT 2011

=========>> EDW_Daily_Sch_DS_step7 Group-2 Job Started at Sat Sep 10 13:48:18 PDT 2011
=========>> EDW_Daily_Sch_DS_step7 Group-2 Has Completed at Sat Sep 10 13:48:58 PDT 2011

Should it wait for only 30 sec for each group?

4

Let me ask you again why are you using backticks? What shell are you using? Try the code without backticks:

#!/bin/bash
echo "=========>> EDW_Daily_Sch_DS_step7 Group-1 Job Started at `date` "
sleep 10 &
sleep 30 &
wait
echo "=========>> EDW_Daily_Sch_DS_step7 Group-1 Has Completed at `date` "
echo ""
echo "=========>> EDW_Daily_Sch_DS_step7 Group-2 Job Started at `date` "
sleep 30 &
sleep 10 &
wait
echo "=========>> EDW_Daily_Sch_DS_step7 Group-2 Has Completed at `date` "
5

It works now by move the & after the backtick. it needs it bcuz the command has more than 1 parm. the sleep ### is only the sample.

`CallWorkFlow wf_one > logs 2>&1` &
`CallWorkFlow wf_two > logs 2>&1` &
`CallWorkFlow wf_three > logs 2>&1` &
wait
`CallWorkFlow wf_four > logs 2>&1` &
`CallWorkFlow wf_five > logs 2>&1` &
`CallWorkFlow wf_six > logs 2>&1` &
wait
6

What is the best way to check for return-code for each CallWorkFLow routine and abort the main job if error returns from any of the call?

7

For the third time, why is everything in backticks? Try removing all the pointless backticks.