I have a requirement to monitor a srver log file realtime for a word 'Started'. Once found, I invoke another program tp perform additional processing.
The code below meets my requirement.
tail -f nohup.out | \
while read line ; do
echo "$line" | grep -i "Started"
if [ $? = 0 ]
then
java AAA.jar $1 $2 >>../AAA/logs/$3
break
fi
done
However, I have another script that has the exact code which greps for 'Shutdown' in the same file 'nohup.out'. This does not capture the word and shutdown does not complete entirely.
Is it because the tail -f command is still open? How can i avoid this?
Well, tail -f can be a problem to manage, but the file is the file. Data can get hung up in the pipe, too. Shell can find strings without calling grep, using case. Logs can be written in blocks from a FILE* buffer. You could poll and just tail a big hunk of file. You could figure how many lines you have checked and just check up-file of there. Really economical and efficient solutions would be in C, where you can mmap() the file as it growns and monitor the content with low overhead just in areas not already checked.
Thanks Pickett. To add more clarity to my query, I only need to look for the word once.
The file 'nohup.out' starts of as an empty file everytime the startup script is executed. Once the word 'STARTUP' is found in the file, AFTER the FIRST OCCURENCE, I no longer need to monitor the file. I can STOP READING the file. --> essentially break out of tail -f , if there is such a thing.
Well, tail -f will die when it discovers it cannot write, once it has something to write, and that is situational. I cannot predict the latency in the pipe.
Quiet grep -q would tell you if it found it and end when it did.
Maybe something like this, if I understand correctly:
while sleep 10; do
grep STARTUP nohup.out > temp.x
lines=`wc -l temp.x`
if [ $lines -gt 1 ]; then
echo STARTUP occured twice in nohup.out
# Do whatever needs doing
fi
done
Wouldn't it make much more sense, esp. as both scripts are managed by you, to write one single script that intercepts both keywords and triggers the respective reaction?
A C/C++/JAVA/PERL/Python level app can keep track of where it has looked and just look at new areas much more easily. Shell has to live with tail -f or some sort of polling the whole file. You might be able to seek in shell using 'tail -####c', where you check between the last and current file size (from ls -l?), then reset the last file size to that current size. If the file size is the same, sleep some more.Rotating logs adds a whole new level of complexity. If you had checked file N up to byte M and it is rotated, you have to detect that and start with Byte M of file N at the new name, then move to the current file. You high water mark is a file number and a byte count.
I do have a single script for both Server Startup & Shutdown with parameters defining the operation & Search Keyword. In both cases, we are monitioring the same file.
There are two scripts - Startup & Shutdown. I added a line at the end of each to execute my script which reads the Server log file.
Once the server starts, we send a message to Syslog and stop reading the log file.
At a later point, when someone decides to stop the server using the shutdown script, it will again invoke my script (which looks for the SHUTDOWN keyword in the log file and send the message to syslog).
---------- Post updated at 11:58 AM ---------- Previous update was at 11:44 AM ----------
I dont have to be concerned about change in file name. Since, my source file name will not change though we have log files rotating.
File size will continually change. during startup, during the time server is up and when the server is shutdown.
Who is writing it when the server is shutdown? Wel, as it shuts down, and then it goes quiescent, I hope!
The idea is that tail -f on a pipe might not process the shutdown due to stuff stuck in the pipe. I think it does, but you seem to have problems, and tail version do vary. The alternative to tail -f if you have tail -####c that seeks, is something like:
seek=0
while :
do
sz=0`ls -l logfile | awk '{print $5}'`
if (( sz == seek ))
then
sleep 1 # or whatever frequency you desire for quiet times
continue
fi
tail -${seek}c $logfile | sed -n '
/pattern1/w file1
/pattern2/w file2
/pattern3/w file3
/pattern4/w file4
'
seek=$sz
done
Since tail reads to EOF and ends, no data is trapped in the pipe. The sed approach allows you to write marker files for each critical message.
Appreciate all your inputs but I am a newbie and a lot of what is suggested is arcane to me. Trying hard to make sense of it and testing it out. Will keep you posted with my final outcome/solution.
Back to previous simple solution, yes there was a glitch. Sorry about that. wc prints the file name, which I forgot. Here is correct version. Try this, see if it works:
while sleep 10; do
grep STARTUP nohup.out > temp.x
lines=`cat temp.x | wc -l`
if [ $lines -gt 1 ]; then
echo STARTUP occured twice in nohup.out
# Do whatever needs doing
fi
done
I believe the $? of grep tells you if it found any. Some grep's have -q for quiet, as $? is the answer. grep -c will give you the line count without the file or wc.
In place of nohup,out, you could have a log for each run appname.YYYY-MM-DD_HH:MM:SS that tells you the start time. Two startups would be two files unless they started the same second (can add .$$ to the end for that!) Each file being the product on one run makes thing easier to understand. The default nohup.out is nice for little adhoc things, and oversights, but normally "(...) >logname 2>&1" keeps nohup.out empty and every command in the parens uses that log unless it redirects to another. In a big shop, it is nice to have a shared log for start and end or abort lines and separate logs for each run, separate dirs for each app.
while sleep 10
do
if [ "" != "$( sed '
1,/STARTUP/d
/STARTUP/!d
q
' nohup.out
)" ] # stops reading at any second STARTUP (file may go on much farther)
then
echo STARTUP occurs twice in nohup.out
# Do whatever needs doing
fi
done
Wel, as it shuts down, and then it goes quiescent, I hope!