Search for logs traced between specific date and time from log file

Shell Programming and Scripting
1

HI,

I want to search for a logs which are trace between specific date and time from logs file.

My logs are generated like this :-

Tue Jun 18 05:00:02 EEST 2013 | file_check.sh| Message:script has files to process.
Thu Jun 20 05:00:02 EEST 2013 | file_check.sh| Message:script has files to process.
Fri Jun 21 05:00:02 EEST 2013 | file_check.sh| Message:script has files to process.
Sat Jun 22 05:00:03 EEST 2013 | file_check.sh| Message:script has files to process.
Sun Jun 23 05:00:02 EEST 2013 | file_check.sh| Message:script has files to process.
Mon Jun 24 05:00:02 EEST 2013 | file_check.sh| Message:script has files to process.
Fri Jun 28 05:00:02 EEST 2013 | file_check.sh| Message:script has files to process.
Sat Jun 29 05:00:02 EEST 2013 | file_check.sh| Message:script has files to process.
Sun Jun 30 05:00:03 EEST 2013 | file_check.sh| Message:script has files to process.
Mon Jul 1 05:00:03 EEST 2013 | file_check.sh| Message:script has files to process.
Tue Jul 2 05:00:02 EEST 2013 | file_check.sh| Message:script has files to process.
Wed Jul 3 05:00:01 EEST 2013 | file_check.sh| Message:script has files to process.
Thu Jul 10 05:00:02 EEST 2013 | file_check.sh| Message:script has files to process.
Thu Jul 16 05:00:02 EEST 2013 | file_check.sh| Message:script has files to process.
Fri Jan 17 04:00:01 EEST 2014 | file_check.sh| Message:script has files to process.
Fri Jan 17 04:05:01 EEST 2014 | file_check.sh| Message:script has files to process. 
Fri Jan 17 05:00:01 EEST 2014 | file_check.sh| Message:script has files to process.
Fri Jan 17 06:05:01 EEST 2014 | file_check.sh| Message:script has files to process.

I want only logs logged between "Fri Jan 17 04:00:00" and "Fri Jan 17 05:50:00"

o/p:-

Fri Jan 17 04:00:01 EEST 2014 | file_check.sh| Message:script has files to process.
Fri Jan 17 04:05:01 EEST 2014 | file_check.sh| Message:script has files to process. 
Fri Jan 17 05:00:01 EEST 2014 | file_check.sh| Message:script has files to process.
2 
$ cat file 
Tue Jun 18 05:00:02 EEST 2013 | file_check.sh| Message:script has files to process.
Thu Jun 20 05:00:02 EEST 2013 | file_check.sh| Message:script has files to process.
Fri Jun 21 05:00:02 EEST 2013 | file_check.sh| Message:script has files to process.
Sat Jun 22 05:00:03 EEST 2013 | file_check.sh| Message:script has files to process.
Sun Jun 23 05:00:02 EEST 2013 | file_check.sh| Message:script has files to process.
Mon Jun 24 05:00:02 EEST 2013 | file_check.sh| Message:script has files to process.
Fri Jun 28 05:00:02 EEST 2013 | file_check.sh| Message:script has files to process.
Sat Jun 29 05:00:02 EEST 2013 | file_check.sh| Message:script has files to process.
Sun Jun 30 05:00:03 EEST 2013 | file_check.sh| Message:script has files to process.
Mon Jul 1 05:00:03 EEST 2013 | file_check.sh| Message:script has files to process.
Tue Jul 2 05:00:02 EEST 2013 | file_check.sh| Message:script has files to process.
Wed Jul 3 05:00:01 EEST 2013 | file_check.sh| Message:script has files to process.
Thu Jul 10 05:00:02 EEST 2013 | file_check.sh| Message:script has files to process.
Thu Jul 16 05:00:02 EEST 2013 | file_check.sh| Message:script has files to process.
Fri Jan 17 04:00:01 EEST 2014 | file_check.sh| Message:script has files to process.
Fri Jan 17 04:05:01 EEST 2014 | file_check.sh| Message:script has files to process.
Fri Jan 17 05:00:01 EEST 2014 | file_check.sh| Message:script has files to process.
Fri Jan 17 06:05:01 EEST 2014 | file_check.sh| Message:script has files to process.

 awk '      BEGIN{
                   split("Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec",M," ")
                   for(i=1;i<=12;i++)Mon[M]=sprintf("%02d",i)
                 }
function dform(v){
                   sub(substr(v,6,3),Mon[substr(v,6,3)],v)
                   gsub(":"," ",v)
                   return mktime(v)
                 }
                 {
                   now = $6" "$2" "$3" "$4
                   if(dform(now)>=dform(start) && dform(now)<=dform(end))
                   print
                 }
     ' start="2014 Jan 17 04:00:00"  end="2014 Jan 17 05:50:00"  file

Resulting

Fri Jan 17 04:00:01 EEST 2014 | file_check.sh| Message:script has files to process.
Fri Jan 17 04:05:01 EEST 2014 | file_check.sh| Message:script has files to process.
Fri Jan 17 05:00:01 EEST 2014 | file_check.sh| Message:script has files to process.

change start and end variable according to your need

3 Likes
3

Thanks Akshay,its what i am looking for....
Could you please explain me how it works...

---------- Post updated at 04:33 PM ---------- Previous update was at 04:10 PM ----------

Hi Akshay,

my system logs are generating in 24hrs date format.so i want to check, if user input start date =04:00:00 & end=05:00:00 then logs also must serch for start=16:00:00 end=17:00:00 i.e logs are generated for PM.

Please suggest change to implement this.

thanks,
rketan

4

This would be fine then

$ cat file
Thu Jul 10 05:00:02 EEST 2013 | file_check.sh| Message:script has files to process.
Thu Jul 16 05:00:02 EEST 2013 | file_check.sh| Message:script has files to process.
Fri Jan 17 16:00:01 EEST 2014 | file_check.sh| Message:script has files to process.
Fri Jan 17 16:05:01 EEST 2014 | file_check.sh| Message:script has files to process.
Fri Jan 17 04:00:01 EEST 2014 | file_check.sh| Message:script has files to process.
Fri Jan 17 04:05:01 EEST 2014 | file_check.sh| Message:script has files to process.
Fri Jan 17 05:00:01 EEST 2014 | file_check.sh| Message:script has files to process.
Fri Jan 17 06:05:01 EEST 2014 | file_check.sh| Message:script has files to process.

awk '      BEGIN{
                   split("Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec",M," ")
                   for(i=1;i<=12;i++)Mon[M]=sprintf("%02d",i)
                 }
function dform(v){
                   sub(substr(v,6,3),Mon[substr(v,6,3)],v)
                   gsub(":"," ",v)
                   return mktime(v)
                 }
                 {
                   now = $6" "$2" "$3" "$4
                   if(dform(now)>=dform(start) && dform(now)<=dform(end) || \
                      dform(now)>=dform(start)+12*3600 && dform(now)<=dform(end)+12*3600)
                   print
                 }
     ' start="2014 Jan 17 04:00:00"  end="2014 Jan 17 05:50:00"  file

Fri Jan 17 16:00:01 EEST 2014 | file_check.sh| Message:script has files to process.
Fri Jan 17 16:05:01 EEST 2014 | file_check.sh| Message:script has files to process.
Fri Jan 17 04:00:01 EEST 2014 | file_check.sh| Message:script has files to process.
Fri Jan 17 04:05:01 EEST 2014 | file_check.sh| Message:script has files to process.
Fri Jan 17 05:00:01 EEST 2014 | file_check.sh| Message:script has files to process.
5

Could you please explain me how it works...

6 
awk '      BEGIN{
                   # Awk BEGIN block is special block in which code is executed before executing actual AWK script

                   # Split string "Jan ... Dec " into array M where delimiter is space

                   split("Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec",M," ")  

                   # Array M holds Month string 
                   # Where M[1] = Jan , M[2] = Feb ...M[12] = Dec

                   for(i=1;i<=12;i++)Mon[M]=sprintf("%02d",i)

                   # Since your log month format is integer so created one more array Mon
                   # Where Mon[Jan] = 01, Mon[Feb] = 02...M[Dec] = 12

                 }
function dform(v){
                 
                   # Assume v is 2014 Jan 10 10:10:10

                   # Substitute 01 for Month Jan 
                   sub(substr(v,6,3),Mon[substr(v,6,3)],v)       

                   # After the execution of above statement v becomes  2014 01 10 10:10:10

                   # Replace colon with space in variable v
                   gsub(":"," ",v)
                   
                   # So now v becomes 2014 01 10 10 10 10
                   
                   # Function mktime returns timestamp in the same form as is returned by systime()

                   # Return timestamp 
                   return mktime(v)

                 }
                 {
                   # now = Year <space> Month <space> Day <space> Time
                   now = $6" "$2" "$3" "$4

                   # Here we are calling function dform

                   # Example dform(now) = dform(2014 Jan 10 10:10:10)

                   # Here goes comparison against input specified in start and end variable 

                   # timestamp+12*3600 is added since you want to search both AM and PM in single querry 
                 
                   # If condition satisfied then it prints line or row

                    if(dform(now)>=dform(start) && dform(now)<=dform(end) || \
                      dform(now)>=dform(start)+12*3600 && dform(now)<=dform(end)+12*3600)
                       print
 
                 }
     ' start="2014 Jan 17 04:00:00"  end="2014 Jan 17 05:50:00"  file