Script in bash that works only some of the time

I ran this script yesterday (in the background)
/usr/bin/nohup myfilelocation/myscriptname.sh &

the script worked perfectly.

i ran it today (also in the background) and just sat there. So i killed it and ran it normally and it worked perfectly.
Anyone suggest why it just sat there and didnt work? but worked when i ran it again right after it worked fine?

partial script below

#!/bin/bash
TZ=date+24
DATE1=`date '+%d%m%y'`
DATE=`date '+%Y%m%d'`

#this bit waits for the file to appear
mytest1=`ls -lart /pathtofile/ | grep -c filename.$DATE.TXT`
while [ $mytest1 -ne 1 ]
do
mytest1=`ls -lart /pathtofile/ | grep -c filename.$DATE.TXT`
while [ $mytest1 -ne 1 ]
sleep 160
done

#this bit waits for the file to finish generating
while [ $mytest15 -ne 1 ]
do
mytest15=`cat /pathtofile/ | grep -c filename.$DATE.TXT | grep TRL | wc -l`
sleep 180
done

goes on to change file name and move it

I don't see why you're performing cat on a directory. Is this a typo? If not, you will not get your intended results. Also, you can grep directly on the file instead of piping the cat output to grep. Read this "useless use of cat".

Other then mentioned, use #!/bin/bash -x and examine the nohup.out during the hang.

If you are on linux system, inotify should provide much better interface for actions on file changes.

Please, if you can, post the entire code and/or explain what are you trying to do so perhaps a better approach could be written.

Hope that helps
Regards
Peasant.

1 Like

sorry that is a typo when written into the forum

mytest15=`cat /pathtofile/filename | grep -c filename.$DATE.TXT | grep TRL | wc -l`

basically i know the file has completed writing when it has the word TRL in the file.

---------- Post updated at 02:10 AM ---------- Previous update was at 02:04 AM ----------

-x i will give that a try.

im positive it could be written a hell of alot better as im still very new.

what the script does is waits for a file to be written to a directory.
when that file arrives, it then waits for it to finish being built by looking for TRL inside the file.
Then it renames the file and moves it.

as i said ran ti yesterday worked fine.
ran it today in the background it didnt see that the file arrived.
so i killed it then ran it
ran it normally...it spotted the file and worked as expected.
i just dont understand why it was in the background it didnt notice that the file was there, and yesterday it worked fine

And even if grep of a directory made sense, the commands at the end of the pipeline:

grep -c filename.$DATE.TXT | grep TRL | wc -l

will always return 0 ; never 1 . The grep -c output will always be a numeric string (that cannot contain the string TRL ) and wc -l reading from an empty pipe will always return 0 .

Therefore, this loop in your code:

#this bit waits for the file to finish generating
while [ $mytest15 -ne 1 ]
do
mytest15=`cat /pathtofile/ | grep -c filename.$DATE.TXT | grep TRL | wc -l`
sleep 180
done

cannot ever complete. It is an infinite loop unless the while loop is failing with a syntax error because $mytest expands to an empty string the first time through the loop yielding a syntax error on:

[ -ne 1 ]

but, since you said it is working successfully when you run it in the foreground, there obviously were not any diagnostic messages printed. So, we have to assume that part of the code you chose not to show us predefined mytest to have the value 1 before the above loop was executed.

If you want to know whether or not a file contains the string TRL , you would not do that with:

mytest15=`cat /pathtofile/filename | grep -c filename.$DATE.TXT | grep TRL | wc -l`

either. You might want to try a simpler loop like:

#this bit waits for the file to finish generating
while ! grep -q TRL /pathtofile/filename
do	sleep 180
done
1 Like

Are you sure it's just sitting there? Are you sure it "worked perfectly"? There's at least a do ... done missing for the second while .

Different approach for waiting "for a file to appear":

while [ ! -f filename.$DATE.TXT ]; do sleep 120; done

And, if one single occurrence of the string "TRL" indicates the file is complete, try

while ! grep -q TRL filename.$DATE.TXT; do sleep 120; done 
1 Like

Many thanks, some brilliant suggestion guys. I'm going to get some sleep then make some improvements and test it again tonight. I'm self taught on scripting but I believe I maybe addicted.

Just an afterthought: if i have gotten correctly what you want to do is 1) wait until a file is there and 2), if it is there wait until "TRL" appears in it.

Such things should be best put into a function, not your main code, if for nothing else than for clarity and readability. The following might not do exactly what you want, take it as a blue print for handling such situations in general:

#! /bin/bash

pCheckForFile ()
{
fIn="/path/to/file/to/check.for"   # alternatively a filename could be provided as argument
# fIn="$1"                         # alternatively a filename could be provided as argument
chText="TRL"                       # text to check for
iRetVal=1                          # value/error status to return

if [ -f "$fIn" ] ; then
     if grep -q "$chText" "$fIn"
          iRetVal=0
     fi
fi

return iRetVal
}



# main()

while ! pCheckForFile ; do
# while ! pCheckForFile /some/file/to/check; do   # this is the alternative call
     sleep 180
done

doSomething

exit 0

You see, "pCheckForFile()" just does the testing, the main part will just use that, wait as long as necessary and then continue. If, for instance, you would want a background job to wait indefinitely and start some processing once the file is there, then sleep again (sort-of daemon) you could change the main part only to have this:

#! /bin/bash

pCheckForFile ()
{
<same as above>
}



# main()

while : ; do
     # if pCheckForFile  /some/file/to/check ; then  # alternative, when a filename is provided as argument
     if pCheckForFile ; then
          doSomeThingHere
     fi
     sleep 180
done

exit 0

I hope this helps.

bakunin

1 Like

Yes you are right a function would be a better approach!