Job Execution in Parallel and Maintain Dependency

Shell Programming and Scripting
1

All,

We need to run multiple jobs in parallel with the dependencies intact.

I was trying to use the same with &, but in vain. Is there is any other way to resolve this issue.

I need to run
Set1
A.sh
B.sh
C.sh

Set2
D.sh
E.sh

Set3
F.sh
G.sh

Jobs Parallely

When Set1 Completes I need to run the below Set4 irrespective of other sets

Set4
AA.sh
BB.sh

When set2 completes I need to run below Set 5 irrespective of other sets

Set 5
CC.sh

When set3 completed I need to run below Set 6 irrespective of other sets

Set6
DD.sh

Once Set 4, 5, 6 completes I need to run set7

Set7
final.sh

How can I make this parallel and set the dependency also?

any help is appreciated.

Regards
Anand M

2

Can anyone help me in this regard?

Any help would be great.

Thanks
Anand

3

Hi ,

bundle the sets(A,B....) to one one scripts and try...
For parallel running normally using "&" ..

for dependency wait command u can use..
Hope this will help

Regards,

4

Hi,

Try something like this...( i have not tested this code)

#!/bin/sh

./Set1.sh &
process_id_set1=`ps -ef | grep ora_q001_DM2XDB | grep -v grep | awk '{print $2}'`
./Set2.sh &
process_id_set2=`ps -ef | grep ora_q001_DM2XDB | grep -v grep | awk '{print $2}'`
./Set3.sh &
process_id_set3=`ps -ef | grep ora_q001_DM2XDB | grep -v grep | awk '{print $2}'`

wait process_id_set1
echo "Set1 Completed -- now start Set4"
./Set4.sh &

wait process_id_set2
echo "Set2 Completed -- now start Set5"
./Set5.sh &

Regards,

5

sorry replace ora_q001_DM2XDB with ur script name

6

Thanks for the reply. I will try the same.

Rgds
Anand

7

what does & mean?
is the script run in background.

8

Yes. I tried with running in background.

9

hi,

if any one of the scripts fails in SET1.how can i restart again.

when i restart, only i want to run the failed job.

if a job in set1 failed i want to update the status as FAILED in database.

after resolving the problem.
when i update the database status as RUN for the corresponding job.
corresponding script has to run automatically.
how can i do that.please advice.

10

#!/bin/ksh

#Set1
(A.sh;B.sh;C.sh)&
set1=$!

#Set2
(D.sh;E.sh) &
set2=$!

#Set3
(F.sh;G.sh)&
set3=$!

#When set1 completes I need to run below Set 4 irrespective of other sets

Set4
(wait $set1;AA.sh;BB.sh)&
set4=$!

#When set2 completes I need to run below Set 5 irrespective of other sets

#Set 5
(wait $set2;CC.sh)&
set5=$!

#When set3 completed I need to run below Set 6 irrespective of other sets

#Set6
(wait $set3;DD.sh)&
set6=$!

#Once Set 4, 5, 6 completes I need to run set7

#Set7
wait $set4
wait $set5
wait $set6
final.sh

#################################

The first line "#!/bin/ksh" forces the script to be run in ksh.
The $! is a ksh builtin returning the last PID run in background

11

if one of the scripts in set1 failed.

how can i restart the only failed job.

suppose A.sh job failed in SET1.
i have to rerun only the failed job.