Run rest of script after parallel steps have completed

Shell Programming and Scripting
1

Hello gurus,

I produce a number of .loc files in parallel depending on number of .csv in the folder

for file in *csv
do
./process.sh $file > $file.loc &
done

then I want to compile all the output from the previous step into a single masterlocfile and then perform the rest of the steps

cat *.loc > masterlocfile

How can I make sure all the .loc files have been completely created (the for loop has ended) before moving onto the next steps including compilation?

I got some answers from google that involves a process id but I will have several different process id from the for loop. Please help.

2

Hi.

On a systerm like:

OS, ker|rel, machine: Linux, 3.16.0-4-amd64, x86_64
Distribution        : Debian 8.9 (jessie) 
bash GNU bash 4.3.30

Best wishes ... cheers, drl

2 Likes
3

Beware that if you have many files that you might hit processing limits (such as number of process per user, total process on the system etc.) so might need to break down your list into smaller chunks.

Is this likely to be the case are we looking at 100 or less files?

Robin