Executing statements inside forloop parallely

Hi All,

I have made a forloop as given below.

if [[ -f "serv_AAA_DS.ldif" && -s "serv_AAA_DS.ldif" ]]; then 
    echo -e "serv_AAA_DS.ldif file created and will be splitted into 8 smaller files for ldapadd \n"
    sh ./file_splitter.sh
    for (( w=1; w <= 8; ++w ))
    do 
    	nohup ldapadd -x -c -h PL0 -p 389 -D "$LDAP_USER" -w $LDAP_PWD -f serv_AAA_DS.ldif_$w > serv_AAA_DS.ldif_$w.log
    done 
    wait
    serv_AAA_DS_count=`cat serv_AAA_DS.ldif | grep '^dn: ' | wc -l`
    echo -e "Count of serv AAA entries present in the Input file is $serv_AAA_DS_count, cross-check the value with total_dm.sh counters & also the check the failures if any in serv_AAA_DS.ldif_*.log \n"
else 
    echo -e "serv_AAA_DS.ldif is empty and will be removed \n"
    rm serv_AAA_DS.ldif
fi

This one runs sequentially waiting for each iteration to finish inside the forloop, I want to execute all the 8 ldapadd parallel and then the code should wait for all the 8 threads to complete, before it moves to the next lines.

Can you let me know what changes to be done on this to achieve the above mentioned.

Thank you in advance.

Assuming your code works correctly, I cannot verify that at the moment:

nohup ldapadd -x -c -h PL0 -p 389 -D "$LDAP_USER" -w $LDAP_PWD -f serv_AAA_DS.ldif_$w > serv_AAA_DS.ldif_$w.log  &

Add an ampersand (& red above) at the end of the line. This works for bash which seems to be the shell you use for this code. If it is another different shell please let us know.

Note:

> serv_AAA_DS.ldif_$w.log

will overwrite the log file every time the script runs, so you lose the content of previous logs. That may or may not be okay.

1 Like

@jim mcnamara - One more query, I have a script.sh that waits for user input to begin. i.e. it has "read" in the script. Now im not able to log the script.sh out as when i do ./script.sh > out.log it doesnt prompt for the input. Also the preferrable mode is to run it as "nohup ./script.sh &" . Can you help me how to log all the outputs from the script ????

The real question is not where the output goes (when you start a process with nohup per default its output goes to ./nohup.out ) but where the input is supposed to come from.

When you start a process on the commandline this commandline is displayed in a terminal. Per default input comes from this terminal (the keyboard) and output goes to this terminal (the screen). Starting a process with nohup cuts this connection to a terminal - actually this is the whole point of nohup : allowing a process to continue even if its terminal is closed ("hangs up").

Bottom line: if you want to start the process with nohup you need something which replaces the terminal as a source of input. That can be some file, a named pipe or whatever - but something has to be there and you need to use redirection to make the process use this instead of waiting indefinitely.

I hope this helps.

bakunin

1 Like

@bakunin - thank you for the help. one more query while the script runs. I'm getting the below

script:

    for (( w=1; w <= 8; ++w ))
    do 
    	nohup ldapadd -x -c -h PL0 -p 389 -D "$LDAP_USER" -w $LDAP_PWD -f serv_AAA_DS.ldif_$w > serv_AAA_DS.ldif_$w.log &
    done 
    wait

To be suppressed:-

nohup: redirecting stderr to stdout
nohup: redirecting stderr to stdout
nohup: redirecting stderr to stdout
nohup: redirecting stderr to stdout
nohup: redirecting stderr to stdout
nohup: redirecting stderr to stdout
nohup: redirecting stderr to stdout
nohup: redirecting stderr to stdout

Is there a way to suppress this.???

nohup: redirecting stderr to stdout

You get that message as UNIX is known to be quite verbous, to say :
As you havent defined anything special for stderr, it will redirect stderr to stdout which is normally your screen...
I would not suggest to redirect to /dev/null as you are in developing stage, you need to be able to read what errors you may get so the best is to redirect in a file, a separate one, or in your log
So your command line could be

nohup ldapadd -x -c -h PL0 -p 389 -D "$LDAP_USER" -w $LDAP_PWD -f serv_AAA_DS.ldif_$w \
> serv_AAA_DS.ldif_$w.log 2>&1 &

Dears,

Please let me know what is the way to print the time taken to complete execution of the below script.

The below will run 8 parallel threads, would like to print the total time taken for all the threads to complete the execution.

    for (( w=1; w <= 8; ++w ))
    do 
    	nohup ldapadd -x -c -h PL0 -p 389 -D "$LDAP_USER" -w $LDAP_PWD -f serv_AAA_DS.ldif_$w > serv_AAA_DS.ldif_$w.log &
    done 
    wait

do date +%s before the loop and after wait - do the math.

Given your shell - which you fail to mention, BTW - is bash , and you don't need more than second accuracy, you can save the SECONDS variable's contents before and after the loop and then calculate the difference. man bash :