# Implement parallel processing **URL:** \ **Category:** Shell Programming and Scripting\ **Created:** [July 18, 2013, 7:34am UTC](https://community.unix.com/t/implement-parallel-processing/331944 "2013-07-18T07:34:20Z")\ **Posts on this page:** 14\ **Page:** 1
**Author:** ![kumarjt](https://community.unix.com/letter_avatar/kumarjt/32/5_5575768a8748004e209b776fc1b2916d.png) [@kumarjt](https://community.unix.com/u/kumarjt)\ **Post date:** [July 18, 2013, 7:34am UTC](https://community.unix.com/t/implement-parallel-processing/331944/1 "2013-07-18T07:34:20Z")
Unix OS : Linux 2.6x Shell type : Korn Hi all , This is a requirement to incorporate parallel processing of a Unix code . I have two pieces of unix code , one of which will act as a parent process . This script will invoke multiple ( say four ) instances of the second script at one go ,and each of these instances will run parallely , exclusive of the status of the other process. How will I implement this logic ? What aspects must I consider while designing this kind of relationship ? Please guide me on this . Thanks Kumarjit ---
**Author:** ![millan](https://community.unix.com/letter_avatar/millan/32/5_5575768a8748004e209b776fc1b2916d.png) [@millan](https://community.unix.com/u/millan)\ **Post date:** [July 18, 2013, 7:36am UTC](https://community.unix.com/t/implement-parallel-processing/331944/2 "2013-07-18T07:36:22Z")
You can use a for loop and put the same job with different parameter to background ---
**Author:** ![kumarjt](https://community.unix.com/letter_avatar/kumarjt/32/5_5575768a8748004e209b776fc1b2916d.png) [@kumarjt](https://community.unix.com/u/kumarjt)\ **Post date:** [July 19, 2013, 6:48am UTC](https://community.unix.com/t/implement-parallel-processing/331944/3 "2013-07-19T06:48:31Z")
No , loops would make the execution sequential, because unless the process within a SINGLE loop ietration completes , the control cannot be passed back to the shell prompt for the next iteration processing to commence. What I want is : Start instance 1 , immediately start instance 2 and do not wait for the exit status of instance 1 . SIMULTANEOUS PROCESSING . Not sequential . Similar to the "tee" command , process input and write to a file at one go. Thanks Kumarjit. ---
**Author:** ![Don\_Cragun](https://community.unix.com/user_avatar/community.unix.com/don_cragun/32/2682_2.png) [@Don\_Cragun](https://community.unix.com/u/Don_Cragun)\ **Post date:** [July 19, 2013, 7:03am UTC](https://community.unix.com/t/implement-parallel-processing/331944/4 "2013-07-19T07:03:33Z")
> [@kumarjt](#): > > No , loops would make the execution sequential, because unless the process within a SINGLE loop ietration completes , the control cannot be passed back to the shell prompt for the next iteration processing to commence. > > What I want is : > Start instance 1 , immediately start instance 2 and do not wait for the exit status of instance 1 . > SIMULTANEOUS PROCESSING . Not sequential . > Similar to the "tee" command , process input and write to a file at one go. > > Thanks > Kumarjit. Is this a homework assignment? If not, please explain why you need to do this. If you're going to ignore our suggestions without understanding what was said, we'll quickly decide to spend our time working with people who will pay attention to our suggestions. Starting a process in the background (AKA starting a process asynchronously) in a loop will allow you to run x instances simultaneously if you go through the loop x times. ---
**Author:** ![rajamadhavan](https://community.unix.com/letter_avatar/rajamadhavan/32/5_5575768a8748004e209b776fc1b2916d.png) [@rajamadhavan](https://community.unix.com/u/rajamadhavan)\ **Post date:** [July 19, 2013, 7:05am UTC](https://community.unix.com/t/implement-parallel-processing/331944/5 "2013-07-19T07:05:30Z")
best place to start is 'man fork' if background process does not suit your need ---
**Author:** ![kumarjt](https://community.unix.com/letter_avatar/kumarjt/32/5_5575768a8748004e209b776fc1b2916d.png) [@kumarjt](https://community.unix.com/u/kumarjt)\ **Post date:** [July 19, 2013, 8:42am UTC](https://community.unix.com/t/implement-parallel-processing/331944/6 "2013-07-19T08:42:59Z")
> [@don cragun](#): > > Is this a homework assignment? If not, please explain why you need to do this. > > If you're going to ignore our suggestions without understanding what was said, we'll quickly decide to spend our time working with people who will pay attention to our suggestions. > > Starting a process in the background (AKA starting a process asynchronously) in a loop will allow you to run x instances simultaneously if you go through the loop x times. Does any of the wordings in my original post give you the feeling that I am seeking help for a HOMEWORK ASSIGNMENT ? It soley a need out of my profession , and if I had even the most basic idea about how to set up asynchronous processing feature (processing at the background or even where to start my search from) , I wouldnt even have taken the pain scribbing a single word for this issue here in this forum . @rajamadhavan : Thanks for the lead , will take it here-on . Regards Kumarjit. ---
**Author:** ![wisecracker](https://community.unix.com/user_avatar/community.unix.com/wisecracker/32/2800_2.png) [@wisecracker](https://community.unix.com/u/wisecracker)\ **Post date:** [July 19, 2013, 8:56am UTC](https://community.unix.com/t/implement-parallel-processing/331944/7 "2013-07-19T08:56:48Z")
First of all have some sensible logic. It is not possible to launch many programs SIMULTANEOUSLY from a single Terminal! However with VERY small increments in time one can launch as many copies of similar or differing scripts and hope that the OS multi(tasking/threading) takes care of the rest. Starter example:- ```nohighlight #!/bin/bash xterm -e /full/path/to/somescript.sh & xterm -e /full/path/to/some_other_script.sh & # NOTE the "&" character! Google is your friend here. x=0 while true do echo "Doing a loop number, $x here..." x=$[( $x + 1 )] done ``` This will launch "somescript.sh" in a separate terminal window and "some\_other\_script.sh" in another terminal window whilst carrying on with its own work! There is a caveat to this method but I suspect you'll never come across it... ---
**Author:** ![rbatte1](https://community.unix.com/letter_avatar/rbatte1/32/5_5575768a8748004e209b776fc1b2916d.png) [@rbatte1](https://community.unix.com/u/rbatte1)\ **Post date:** [July 19, 2013, 10:55am UTC](https://community.unix.com/t/implement-parallel-processing/331944/8 "2013-07-19T10:55:26Z")
Why spin up another terminal? You can background the processes with this:- ```nohighlight counter=1 until [$counter -ge 10] do some_script ${counter} & ((i=$i+1)) done wait # Script will not continue until they all finish ``` This, of course, assumes that some\_script can recognise the counter as meaning something. If you just have five distinct scripts to run, you could just:- ```nohighlight first_script & second_script & third_script & fourth_script & fifth_script & wait # Script will not continue until they all finish ``` Do either of these options help? Robin Liverpool/Blackburn UK ---
**Author:** ![wisecracker](https://community.unix.com/user_avatar/community.unix.com/wisecracker/32/2800_2.png) [@wisecracker](https://community.unix.com/u/wisecracker)\ **Post date:** [July 19, 2013, 11:06am UTC](https://community.unix.com/t/implement-parallel-processing/331944/9 "2013-07-19T11:06:56Z")
Hi rbatte1... > [@](#): > > Why spin up another terminal? Why indeed, except that with these extra terminals running one can keep a watchful eye on each \_child\_ as it \_runs\_. Also you can see immediately when the script has done its all, as it will autoclose, \_no\_longer\_to\_be\_seen\_... ;o) No other reason than that. You can also give each terminal its own number that shows in its header bar so you know how each task is progressing... All good fun... ;o) ---
**Author:** ![Corona688](https://community.unix.com/letter_avatar/corona688/32/5_5575768a8748004e209b776fc1b2916d.png) [@Corona688](https://community.unix.com/u/Corona688)\ **Post date:** [July 19, 2013, 11:32am UTC](https://community.unix.com/t/implement-parallel-processing/331944/10 "2013-07-19T11:32:03Z")
> [@kumarjt](#): > > Does any of the wordings in my original post give you the feeling that I am seeking help for a HOMEWORK ASSIGNMENT? Your question is oddly specific for someone who doesn't know how to program in this language, so, yes. You can run a process in the background in shell with ```plaintext programname & ``` ---
**Author:** ![Corona688](https://community.unix.com/letter_avatar/corona688/32/5_5575768a8748004e209b776fc1b2916d.png) [@Corona688](https://community.unix.com/u/Corona688)\ **Post date:** [July 19, 2013, 11:33am UTC](https://community.unix.com/t/implement-parallel-processing/331944/11 "2013-07-19T11:33:19Z")
> [@wisecracker](#): > > Why indeed, except that with these extra terminals running one can keep a watchful eye on each \_child\_ as it \_runs\_. Also you can see immediately when the script has done its all, as it will autoclose, \_no\_longer\_to\_be\_seen\_... ;o) Very good point! This is not suitable for all tasks however, and not everyone has xterm, so it's important to point out that this is optional. ---
**Author:** ![rbatte1](https://community.unix.com/letter_avatar/rbatte1/32/5_5575768a8748004e209b776fc1b2916d.png) [@rbatte1](https://community.unix.com/u/rbatte1)\ **Post date:** [July 19, 2013, 11:33am UTC](https://community.unix.com/t/implement-parallel-processing/331944/12 "2013-07-19T11:33:31Z")
This might be a batch process request though...... I'm still unclear what's really wanted. ---
**Author:** ![kumarjt](https://community.unix.com/letter_avatar/kumarjt/32/5_5575768a8748004e209b776fc1b2916d.png) [@kumarjt](https://community.unix.com/u/kumarjt)\ **Post date:** [July 20, 2013, 4:27am UTC](https://community.unix.com/t/implement-parallel-processing/331944/13 "2013-07-20T04:27:28Z")
@ wisecraker : I am using the method specified by millan ( for loop , and within each loop execute the command as background proces ) , and it works fine , but after all the iterations of the for loop is complete , the control does not come back to the shell prompt so that any subsequent conmmand could be executed . Please tell me how to bring back the shell prompt again at the end of the for loop. Thanks ---
**Author:** ![wisecracker](https://community.unix.com/user_avatar/community.unix.com/wisecracker/32/2800_2.png) [@wisecracker](https://community.unix.com/u/wisecracker)\ **Post date:** [July 20, 2013, 5:42am UTC](https://community.unix.com/t/implement-parallel-processing/331944/14 "2013-07-20T05:42:04Z")
Here is an example... The launch code:- ```nohighlight #!/bin/bash --posix var="" for n in $( seq 1 2 ) do xterm -e ~/childmultitask.sh & done echo "Done!" echo "You are still in your original calling script!!!" echo "Your background srcipts are running now so......" read -p "Press to go back to the terminal:- " -e var exit ``` This is the child process:- ```nohighlight #!/bin/bash --posix x=0 while true do echo -n "$x " x=$[( x + 1 )] sleep 1 done ``` Attached is a snapshot of what was done.