Logs between two time stamp

Shell Programming and Scripting
1

I am creating log monitoring script and stuck up to get the logs between two time stamp.

can you please help me to create the script to get the logs between two time stamp, for example, I need the complete logs between

 # Time: 150328  1:30:10 and # Time: 150328 19:10:57

OS : Cent OS 6.x
Date format in the log :

 `date "+%y%m%d  %H:%M:%S"`

Sample logs:

# Time: 150328  1:30:10
# User@Host: testuser[testuser] @ test.domain.com [192.168.1.1]
# Query_time: 20.00009 Lock_time: 0.000063 Rows_sent: 9  Raw_del: 1
SET timestamp=1234567890;
xyz-set order number =  xxxx. phone number = null, phone number = test, email_ address=null street address place=null direction=gps test=query email address=invalidate
code=testzero first email from test user zip = 0000 house number = 0000 zip_house location level_of_service error invalid test_test123 floor=null room=null status_code
buillding_address floor-bull where id 000 test 123:
# Time: 150328 11:15:15
# User@Host: testuser[testuser] @ test.domain.com [192.168.1.1]
# Query_time: 20.00009 Lock_time: 0.000063 Rows_sent: 9  Raw_del: 1
SET timestamp=1234567890;
xyz-set order number =  xxxx. phone number = null, phone number = test, email_ address=null street address place=null direction=gps test=query email address=invalidate
code=testzero first email from test user zip = 0000 house number = 0000 zip_house location level_of_service error invalid test_test123 floor=null room=null status_code
buillding_address floor-bull where id 000 test 123:
# Time: 150328 19:10:57
2

Please, try:

perl -nle 'print if /150328\s+1:30:10/../150328\s+19:10:57/' zenkarthi.file
3

it is printing all the logs. Also, the idea to run the script for every 20 mins and collect the past 15 mins logs....

4

Given your sample is representative, i.e. the last line has a time stamp entry, and you can have a file somewhere holding the last time stamp, try

awk 'FNR==NR {START=$0; next} END {print > "lasttime"} $0 ~ START, EOF ' lasttime logfile
1 Like
5

Thanks RudiC. can you explain little more on this please?

As I mentioned, the script has to run every 20 mins to collect the past 15 mins logs from the log file...for example, if script is running at 8:00, it has to collect the logs from 7:45 to 8:00 if the log file has some update on that time...

6

The last line, presumedly holding a time stamp, is stored in a separate file called "lasttime". At the next run, this time stamp is read, and the log file is parsed for it to start printing until end-of-file. For simplicity, it's always starting with that line; it doesn't subtract five minutes from a 20 min intervall. Aside, it would be difficult to match a time stamp that is exactly 15 min ago...

7

oh Sorry, the last line is not holding the time stamp, by mistake I copied next line starting word, it has the time stamp...is any other way I can try instead of adding grep line in the script.

awk 'FNR==NR {START=$0; next} $0 ~ START, EOF ' lasttime log
grep -i "# Time:" log | tail -1  > lasttime
8

ad-lib and untested: you could save any (last) line - the next run would resume from there.
Or, replace the END section with /# Time:/ {timeline=$0} END {print timeline > "lasttime"}

1 Like
9

Perfect! It is working exactly for my requirements. Thanks RudiC for your help!