How to get script to wait until status is true before continuing?

Shell Programming and Scripting
1

I'm extremely new to shell scripting so I apologize for the crudeness of my descriptions. I am editing a script that will write files (e.g. Job0_A.com, Job1_A.com, etc.) and then run them through a program called gaussian (computational chemistry program). The script will then take the output files (e.g. Job0_A.log, Job1_A.log, etc.) and run them through a fortran program to do further optimizations. Since I am on a multi-user cluster, I have edited the script to run batch files. It seems to have worked but there is a problem. The script will not wait until the .log file has been created (I am running large systems so it make take several hours for the .log to be created) and will just go on to the next job. How do I make the script wait until

ls | grep 'Job'$num'_A.log >& /dev/null

is true? Also, when I run the script I get a file called $ with nothing in it. I searched the script to see if I made a mistake that would cause this but I can't find anything.

2

One easy way is to have the first process create the file in a different path (name, extension, directory) of the same device, and when done and happy, move it so the next process can start. Usually, scripts start the next process right then, so it is not an issue. If the second process needs multiple processes from the first set of processes, it waits for one file and then waits for the next until all are present. Serial data can even be piped from one program to the next without a flat file, or with the flat file produce by tee on the pipeline. However, scientific array problems want to mmap() the whole file into VM for processing, so pipes need not apply!

#!/bin/bash
 
while :
do
 if [ -f file1 -a -f file2 -a -f file3 ]
 then
  break
 fi
 
 sleep 1
done
 
process2 file1 file2 file3
1 Like
3

Thanks for the reply! Again, I'm very new to this so I apologize, but I don't understand some of the code. In

if [ -f file1 -a -f file2 -a -f file3 ]

what does the -f and -a mean? Also is the "1" after sleep the number of seconds until the loop restarts?
I only need two files from the first process and they must both be present at the same time for the next process to work. Do you think this would work:

ls | grep 'file1' 'file2' >& /dev/null
while:
do 
    if ( $status == 0 )
    then
       break
    fi
    sleep 1
done

process2 file1 file2

Again, I thank you for your help!

4

That 'if' looks a lot like C shell, what shell are you using?

5

-f is file from man test, as '/bin/[' is a hard link to '/bin/test' and some shells mimic it with builtins.

6

Okay thanks! I'll give it a try.

It is the C shell. The script I am editing has /bin/csh -f.

7

That would have been nice to know. The code you've been given so far will not work in C shell.

8

I apoligize. As I said I am very new to this. I didn't know it was different for different shells, but I will be sure to include that information next time.
I wanted to let you all know that I got it to work with the following code:

while (0 == 0)
   ls | grep $name >& /dev/null
   if ($status == 0) then
      break
   else
      sleep 1
   endif
end

A buddy of mine told me that there problems may arise with just saying:

while (0 == 0)

and that something like:

x=0
while ($x == 0)

would be better. Can anyone explain why?

9

You can test many ways in sh, ksh, bash using different delimiters: [] () (()) [[]]

  • () is same as cmd, a subshell, and the return (exit) 0 is true.
  • Basically, [ is test, see man test.
  • ksh/bash (()) are very nice for math, logic, numbers and bits, with operators + - * / % && & || | () and such like C.
  • bash [[]] allows regex testing.

Using builtins is much cheaper than fork/exec a command or two, and the test bits are now built in, so to find a file, it is far simpler to [ -f file ] than ( ls | grep file ), which is two fork/exec and a pipe. For the former, the shell just does an access() call.

Polling files is a sad way to flow control jobs. It harkens back to the days of tape. The shell has better ways to keep track of children, e.g., (proc1&proc2&proc3&wait;proc4), and many times pipes make flow control and intermediate flat files unnecessary while reducing latency by pipeline parallelism: proc4 <(proc1) <(proc2) <(proc3)

10

Thank you for the information. You are very helpful. I will give this a try. Someone told me about waitpid and so I gave that a try, but I found out that the process is not a child process. I didn't know this before or I would've provided that info. Do you have a suggestion for a book I can buy or a web reference to learn a lot of this stuff? I feel I ask to many questions on here, and you have been very helpful, but I feel I'm in over my head. I'm a chemist by training so this is all new to me. So if you know of a good resource let me know.

11

Well, usually you can arrange to be parent to things you want to monitor. Basic sh, ksh, bash have wait, which wait for all unless you provide one or more pid. Man Page for wait (all Section 1) - The UNIX and Linux Forums Wait will set $? according to the exit of the waited child, so I like to make a pid list and use a for loop to wait for and report on each. After all, if you are waiting for all N, it does not matter which you sense first. If you want to get the exit time, you can create a wait on each in the background, and wait on that list.

date "+%Y-%m-%d %H:%M:%S ${0##*/) ($$) Start apps."
( # subshell only waits for subshell's children
for prog in prog1 prog2 prog3
do
( $prog  ; date '+%Y-%m-%d %H:%M:%S ${0##*/) ($$) Exit $! $prog." )&
done
wait
) >logfile 2>&1   # subshell also collects all stdout, stderr for log