# Parallel processing and error checking **URL:** \ **Category:** Shell Programming and Scripting\ **Created:** [February 15, 2016, 7:27pm UTC](https://community.unix.com/t/parallel-processing-and-error-checking/358564 "2016-02-15T19:27:46Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![wahi80](https://community.unix.com/letter_avatar/wahi80/32/6_22734026db12803ebb3e059622ac3534.png) [@wahi80](https://community.unix.com/u/wahi80)\ **Post date:** [February 15, 2016, 7:27pm UTC](https://community.unix.com/t/parallel-processing-and-error-checking/358564/1 "2016-02-15T19:27:46Z")
Hi, I need to run multiple scripts in parallel. The scripts needs to ensure that x scripts are always running. I trigger 6 jobs in parallel, as soon as one finishes it triggers the next one, but id something fails it should stop processing any further jobs. I was able to get hold of a script which helped me in first part, but not sure how to handle the error tracking. I do not have GNU parallel ☹ ```nohighlight #!/bin/ksh PROCS=6 SLEEP=20 while read LINE do while true do NUM=$(jobs | wc -l) echo NUM=$NUM if [$NUM -lt $PROCS] then stuff "$LINE" & break else sleep $SLEEP fi done done < textfile # wait for ALL remaining processes wait ``` So above ensures 6 processes are always running. I want a way to find out if one of the process is failed, then do not trigger anymore jobs. Thanks ---
**Author:** ![drl](https://community.unix.com/user_avatar/community.unix.com/drl/32/542_2.png) [@drl](https://community.unix.com/u/drl)\ **Post date:** [February 15, 2016, 9:20pm UTC](https://community.unix.com/t/parallel-processing-and-error-checking/358564/2 "2016-02-15T21:20:10Z")
Hi. This is a feature of GNU ` parallel` : ```plaintext --halt-on-error <0|1|2> --halt <0|1|2> -H <0|1|2> (-H will be retired 20120122) 0 Do not halt if a job fails. Exit status will be the number of jobs failed. This is the default. 1 Do not start new jobs if a job fails, but complete the running jobs including cleanup. The exit status will be the exit status from the last failing job. 2 Kill off all jobs immediately and exit without cleanup. The exit status will be the exit status from the failing job. ``` See `man parallel` for details. If GNU parallel is not in your repository, see [GNU Parallel - GNU Project - Free Software Foundation](http://www.gnu.org/software/parallel/) Best wishes ... cheers, drl ---
**Author:** ![wahi80](https://community.unix.com/letter_avatar/wahi80/32/6_22734026db12803ebb3e059622ac3534.png) [@wahi80](https://community.unix.com/u/wahi80)\ **Post date:** [February 16, 2016, 12:20pm UTC](https://community.unix.com/t/parallel-processing-and-error-checking/358564/3 "2016-02-16T12:20:03Z")
I have mentioned in my post I do not have GNU parallel ---------- Post updated at 01:20 PM ---------- Previous update was at 09:02 AM ---------- Is there a way to capture pids of each process triggered in array and check for the exit status. For non-0 exit status I will stop triggering jobs ---
**Author:** ![tange](https://community.unix.com/letter_avatar/tange/32/6_22734026db12803ebb3e059622ac3534.png) [@tange](https://community.unix.com/u/tange)\ **Post date:** [February 16, 2016, 2:02pm UTC](https://community.unix.com/t/parallel-processing-and-error-checking/358564/4 "2016-02-16T14:02:39Z")
> [@wahi80](#): > > I have mentioned in my post I do not have GNU parallel If you are allowed to run your own perl script you are allowed to run GNU Parallel. See: [\_: Excuses for not installing GNU Parallel](http://oletange.blogspot.dk/2013/04/why-not-install-gnu-parallel.html) ---
**Author:** ![wahi80](https://community.unix.com/letter_avatar/wahi80/32/6_22734026db12803ebb3e059622ac3534.png) [@wahi80](https://community.unix.com/u/wahi80)\ **Post date:** [February 16, 2016, 3:11pm UTC](https://community.unix.com/t/parallel-processing-and-error-checking/358564/5 "2016-02-16T15:11:24Z")
HI, I have worked with parallel in earlier projects. In my current project the Admins will not install GNU parallel. So is there a way out with the code I have written? Thanks ---
**Author:** ![tange](https://community.unix.com/letter_avatar/tange/32/6_22734026db12803ebb3e059622ac3534.png) [@tange](https://community.unix.com/u/tange)\ **Post date:** [February 16, 2016, 3:25pm UTC](https://community.unix.com/t/parallel-processing-and-error-checking/358564/6 "2016-02-16T15:25:43Z")
> [@wahi80](#): > > In my current project the Admins will not install GNU parallel. I am confused now. Did you read [\_: Excuses for not installing GNU Parallel](http://oletange.blogspot.dk/2013/04/why-not-install-gnu-parallel.html)? And did you understand that you do \*not\* need admins to install GNU Parallel as long as you are allowed to run your own scripts (excuse #3)? If yes: Please elaborate why none of the excuses apply to you. ---
**Author:** ![MadeInGermany](https://community.unix.com/letter_avatar/madeingermany/32/6_22734026db12803ebb3e059622ac3534.png) [@MadeInGermany](https://community.unix.com/u/MadeInGermany)\ **Post date:** [February 16, 2016, 3:55pm UTC](https://community.unix.com/t/parallel-processing-and-error-checking/358564/7 "2016-02-16T15:55:07Z")
Run a controll shell that can write an error file: ```nohighlight errfile="errorfile" rm "$errfile" while read LINE && [! -f "$errfile"] do ... while [! -f "$errfile"] do ... (stuff "$LINE" || echo "bad exit status" > "$errfile") & ... ``` ---
**Author:** ![wahi80](https://community.unix.com/letter_avatar/wahi80/32/6_22734026db12803ebb3e059622ac3534.png) [@wahi80](https://community.unix.com/u/wahi80)\ **Post date:** [February 16, 2016, 8:31pm UTC](https://community.unix.com/t/parallel-processing-and-error-checking/358564/8 "2016-02-16T20:31:37Z")
> [@tange](#): > > I am confused now. Did you read [\_: Excuses for not installing GNU Parallel](http://oletange.blogspot.dk/2013/04/why-not-install-gnu-parallel.html)? And did you understand that you do \*not\* need admins to install GNU Parallel as long as you are allowed to run your own scripts (excuse #3)? > > If yes: Please elaborate why none of the excuses apply to you. Admins had blocked access to blogs from work location.Got home and read point 3.Will try it out tomorrow