Script to read log file

UNIX for Advanced & Expert Users
1

Hi,

Im looking for a shell script which will search for a particular string in a log file as below scenario

  1. I need to run URL http://localhost/client/update?feedid=200 in shell script at(eg)4:00 PM which will not take more than 15 mins to complete.

  2. After 15 mins i need to check the log from 4:00 PM to 4:15 PM.

  3. Need to check logs file for 2 strings (a) Feed id=200 (b) Item=Product example, if these 2 string matches i need to check for "Error" string.

  4. If 3 string matches i will send a mail stating "Failed"

  5. If string "Error" is not found in last 15 mins log(only string a & b should match) then i will execute another 6 URLS and atlast will trigger mail stating "Success"

  6. Is this above sequence possible in shell script? Please guide me and my script is below - Im really stuck in comparing string and checking last 15 mins log.

  7. Log with Error

[INFO 12-03-01:04:10:25] Starting FeedRunner for feed: 200  Product example Items 
[INFO 12-03-01:04:12:40] Error FeedRunner for feed: 200 Product example Items
[INFO 12-03-01:04:20:25] Stopping FeedRunner for feed : 200  Product example Items Feed time taken 231743
  1. Log without Error
[INFO 12-03-01:04:10:25] Starting FeedRunner for feed: 200  Product example Items 
[INFO 12-03-01:04:20:25] Stopping FeedRunner for feed : 200  Product example Items Feed time taken 2743
  1. Script
#!/bin/sh

START=$(date +"%y-%m-%d:%T")
echo $START
curl "http://localhost/mcfeeds/runfeed?feedid=200"
END=$(date "%y-%m-%d:%T")
echo $END

cat /path/logfile.log | sed -n '/$START/,/$END/ p' > output.txt

grep '200'&'Product example'&'Error' output.txt

Thanks
Paulwintech

2

I don't think there's any point making 'start' and 'end' dates like you have there. It'll only take a fraction of a second to get the feed, and those dates aren't useful in comparing anything in the log file.

What's your system? What's your shell? On Linux, GNU awk has features allowing you to get epoch time from timestamps, which is very handy for comparing how long ago things happened...

3

Hi Corona688,

Thanks you!!! Please let me know any example to compare and to search in log file.

Regards
Paulwintech

---------- Post updated at 11:05 AM ---------- Previous update was at 10:43 AM ----------

Hi,

Is my script right now? Please help me if im wrong

#!/bin/bash

SUBJECT="SET-EMAIL-SUBJECT"
EMAIL="admin@somewhere.com"

START=$(date +"%y-%m-%d:%T")
echo $START
curl "http://localhost/mcfeeds/runfeed?feedid=200"
END=$(date "%y-%m-%d:%T")
echo $END

cat /path/logfile.log | sed -n '/$START/,/$END/ p' > output.txt

if [egrep -i '200|Product example|Error' output.txt]
then 
/bin/mail -s "$SUBJECT" "$EMAIL" < "Feed Failed"
else
curl "URL1"
curl "URL1"
curl "URL1"
curl "URL1"
curl "URL1"
curl "URL1"
/bin/mail -s "$SUBJECT" "$EMAIL" < "Feed Success"
fi
4

Please tell us more about the logfile /path/logfile.log .
Does the program append to the log every time it is run or is the log created from scratch?
If the log is appended you could consider adding your own entries to the log containing the values of your $START and $END variables such that you can find them with "sed".

Please reply to the above request: What Operating System and version do you have and what Shell are you using?

There are multiple logic and syntax errors in your most recent script.

5

I asked you what your system was.

This was not an off-hand question. Different systems have very different ways of working with dates. Any answer I give you right now has good odds of being useless to you, since it's for the wrong system.

Please answer my questions so I can help you.

6

Hi Corona688/methyl,

Operating System and version
CentOS release 5.5 (Final)

Shell are you using?
#!/bin/bash

Does the program append to the log every time it is run or is the log created from scratch?
--Yes it updates the log every time it runs

Thanks
Paulwintech

7

Best approach is to make your own entries to the log from shell. One before the run and one after the run. Stick to the syntax of the logfile itself for the left hand side of the logfile entry, but include your own unique start and end string to your special lines in order to make it possible to use your original "sed" syntax.
I would put your unique search string in the text portion of the message and use only alphanumeric characters. You just need to include the date but in numeric yyyymmddhhmmss format and something to distinguish start from end.
For example

[INFO 12-03-01:04:10:24] S20120301041024
[INFO 12-03-01:04:10:25] Starting FeedRunner for feed: 200  Product example Items 
[INFO 12-03-01:04:12:40] Error FeedRunner for feed: 200 Product example Items
[INFO 12-03-01:04:20:25] Stopping FeedRunner for feed : 200  Product example Items Feed time taken 231743
[INFO 12-03-01:04:10:26] E20120301041026