Collecting all lines between two time stamp from the log

Can you help me to collect the entire logs between two time stamp. The below awk command collecting the logs only if the line has time stamp.

awk '$0>=from && $0<=to' from="150318 23:19:04" to="150318 23:55:04" log file

150318 23:19:04 logentries 
150318 23:29:04 logentries 
150318 23:39:04 logentries 
150318 23:49:04 logentries 
150318 23:55:04 logentries 

I am looking the awk command or script to collect the entire lines between two time stamp like below


150318 23:19:04 logentries 
150318 23:29:04 logentries 
150318 23:39:04 logentries 
logentries 
logentries 
logentries 
150318 23:49:04 logentries 
150318 23:55:04 logentries 

Change the awk script to

'$0>=from {pr=1} $0>to {pr=0} pr'

Thanks for the reply!

awk '$0>=from && $0<=to' 

and

'$0>=from {pr=1} $0>to {pr=0} pr' 

both output looks the same, both are not printing the lines which do not have the time stamp.

The problem is that the string comparison is not precise enough.
So before the comparisons one needs to ensure that there is a time stamp.

awk '/^[0-9][0-9][0-9][0-9][0-9][0-9] [0-9][0-9]:[0-9][0-9]:[0-9][0-9] / {if ($0>=from) {pr=1} if ($0>to) {pr=0}} pr' from="150318 23:19:04" to="150318 23:55:04" logfile
1 Like
sed -n '/150318 23:19:04/,/150318 23:55:04/{p}'

awk equivalent:

awk '/150318 23:19:04/,/150318 23:55:04/' file
150318 23:19:04 logentries 
150318 23:29:04 logentries 
150318 23:39:04 logentries 
logentries 
logentries 
logentries 
150318 23:49:04 logentries 
150318 23:55:04 logentries

But all of these would work only if the exact time stamps are known and given. Should that not be the case, you'll need to read every line and interpret/calculate the time values before comparing to the lower and upper limits.

Thank you all!

@MadeInGermany - your suggestion is working if the time value is lower to higher but if the time value is higher to lower then the output is completely different .

RudiC , The idea, script to collect the logs for every 45 mins and check the errors automatically.

I found the below perl script in other post, it is working perfectly for my requirements, since "from" and "to" time is unknown, can you help me to create the Perl script to take the current time for "from" and "-30" mins for "to".

perl -ne 'BEGIN {$from="150318 23:40:04" ; $to="150319  1:10:40"; $from=`date -d "$from" +%s`; $to=`date -d "$to" +%s`;} $r=`date -d "$1" +%s` if /^(.*?)\[/; print if ($r <= $to) && ($r >= $from);' file

looking similar in the perl

to=`date "+%y%m%d %H:%M:%S"`
from=`date "+%y%m%d %H:%M:%S" -d "-30 minutes"`

My perl is quasi non-existent, but as you are running date from within it, try date +%s for now, and date -d -30min +%s for the other ?

I tried the date in the perl and it showing invalid option.

I went back and read your first post and this will not do what you said your requirements are. Nevertheless, just so you believe me I will convert it to what you asked.

perl -ne '
    BEGIN{$from=qx(date -d -30min +%s); $to=qx(date +%s)}
    $r=qx(date -d "$1" +%s) if /^(.*?)\[/;
    print if ($r <= $to) && ($r >= $from);
' file

Even, if modified to accept a current time for to and 30 minutes ago for from , the condition if will not match any entries in your posted log. Substituting it for if /^(\d{6}\s+\d{2}:\d{2}:\d{2})/ it will match your time stamp format, but it will not print properly those entries in between that do not contain a time stamp.

Please, try the following:
Save as entries.pl and run as perl entries.pl log.txt

#!/usr/bin/perl

# this is not tested since I do not have any valid log files.
use strict;
use warnings;
use POSIX qw(strftime);

my $now = time;
my $ago = $now - (30*60);
my $from = strftime('%y%m%d %T', localtime($ago));
my $to = strftime('%y%m%d %T', localtime($now));

while(<>) {
    print if /$from/../$to/;
}
1 Like

Perfect , Thank you Aia for your help! Perl -ne syntax working well.

for the perl entries.pl log.txt returning to prompt, since it is working only if the exact /$from/ timestamp matching in the log.txt