Need Help With Parallel Processing

Shell Programming and Scripting
1

Hi

I am looking for some kind of feature in unix that will help me write a script that can invoke multiple processes in parallel. And make sure that the multiple parallel processes complete successfully before I proceed to the next step.

Someone suggested something called timespid or timepid or something that sounds like that, and recommended that it might help me accomplish what Im looking for.

Can someone help me with this?

Thanks!
A Beginner

2

If you just want to start processes you could use the background process mechanism:

command &

will start the process "command" but will continue processing of the program issuing it after it is started. You can find out the background jobs PID with the "jobs" internal shell command and look in a loop if all started background processes have ended or not.

The following is a sketch - note, that the output of "jobs" is a little different from one system to another. This was written on AIX 5.3, maybe you need to change the regexps a bit to make them fit.

#! /bin/ksh
typeset -i iPID=0

some_command &
iPID=$(jobs -l | sed -n 's/^\[[0-9]*\]  *+  *\([^ ]*\) .*/\1/p')

while [ $(ps -fe | grep -c $iPID) -gt 1 ] ; do
     # wait for process to end - process is still running
     sleep 1
done

print - "process has finished"

exit 0

If you need to exchange messages with the subprocesses you will have to use the Korn Shells Coprocess facility: a coprocess is a background process just like above, but one that can communicate with the main process through two file descriptors: &4 and &5. There are special commands ("read -p" and "print -p") to communicate between the processes. You might want to read some documentation first, because building such a coprocess is a bit complicated at the beginning.

I hope this helps.

bakunin

3

It is straightforward to run multiple processe in parallel, just run them in the background using "&".

# invoke job 1 in the background
job1 & 
# invoke job 2 in the background
job2 &
# wait for background processes to complete
wait
# any additional processing

You can also wait for a specific process to finish before continuing, see the wait command on your shell's man page.

4

bakunin, a much easier method: iPID=$(jobs -p).

Also easier: while ps -p $iPID > /dev/null.

5

Also just get the PID by iPID=$! after starting the background process, and wait $iPID to wait on it without the need for a wasteful loop, with or without sleeps.

6

:slight_smile: I will try these suggestions.

Is anyone aware of anything called timepid or timespid or something like that? I tried Google, but no useful results.

7

I sort of assumed you meant waitpid