waiting on jobs in bash, allowing limited parallel jobs at one time, and then for all to finish

Shell Programming and Scripting
1

Hello,

                                                 I am running GNU bash, version 3.2.39\(1\)-release  \(x86_64-pc-linux-gnu\).  I have a specific question pertaining to waiting on jobs run in  sub-shells, based on the max number of parallel processes I want to  allow, and then wait for the remaining sub-shell jobs to finish before  the next step is executed in the pipeline \(if I am making proper sense  here\)..

Essentially,my pseudo code looks like this:

MAX_PROCS=3
    for (( k = 0 ; $k < $kmerlen ; k += 1 ))
    do
    (
     ### Running a perl script here for each k (this script is a memory hog)...
    )&
    while [ $(ps -e | grep 'perlScriptAbove' | grep -v grep | wc -l) -gt ${MAX_PROCS} ] ; 
    do
             wait
    done

    done

\#\#\#wait &lt;- works fine without this wait, but I need all kmerlen jobs to finish first to proceed to the next part of the pipeline
\#\# Run the rest of the pipeline...

The first wait statement in the while loop works fine spawning 3 jobs, but when I use the next wait statement, that property is lost, and the number of sub-shells spawned are equal to my kmerlen.

Thanks for any pointers that you can provide.

2

BASH allows you to wait for one specific process via wait PID but doesn't allow you to wait for N random processes. So you'll need to keep a list, and wait in order.