Interactive filters for log file (beginner)

Hello,

I have a software which log all entry from internet and save it in text file.

For example (fake):

 10:02:23:124.id_0000.1:"blalba"
10:05:26:124.id_0000.1:"blalba"
10:10:32:124.id_0000.1:"blalba"
11:32:36:124.id_0000.1:"blalba"
11:33:49:124.id_0000.1:"blalba"
11:36:23:124.id_0000.1:"blalba"
12:10:21:124.id_0000.1:"blalba"

I just need to enter two time limit (start and end time).

For example with start = "11:30:00" and end = "11:40:00, should return:

11:32:36:124.id_0000.1:"blalba"
11:33:49:124.id_0000.1:"blalba"
11:36:23:124.id_0000.1:"blalba"

I tried some script with awk, but, I'm beginner and I need your help :slight_smile:

Thanks a lot for your response,

Acid

[edit] the log file can be huge (more than 100 000 lines).

awk ' /^11:[34][0-9]:[0-9][0-0]/ {print}' filename
start_line=`grep -n -m 1 "^11:30:00" filename | awk '{print $1}'`
end_line=`grep -n -m 1 "^11:40:00" filename | awk '{print $1}'`
sed '"$start_line","$end_line"p' filename

Or for something more re-usable (DATA section for illustration, you'd have to open the file for reading in a programmed script.

#!/usr/bin/perl

@divs=qw(hour min sec milis);
@start{@divs} = $ARGV[0]=~/(\d+)/g;
@end{@divs} = $ARGV[1]=~/(\d+)/g;
while(<DATA>){
    chomp;
    @time{@divs} = $_ =~/(\d+):/g;
    if ((after(\%time, \%start)) && (! after(\%time, \%end))){
        print "$_\n";
    }
}
sub after{
    my ($time,$limit)=@_;
    for (@divs){
        if ($time->{$_} < $limit->{$_}){
            return 0;
        }
        elsif ($time->{$_} > $limit->{$_}){
            return 1;
        }
    }
    return 1;
}
__DATA__
10:02:23:124.id_0000.1:"blalba"
10:05:26:124.id_0000.1:"blalba"
10:10:32:124.id_0000.1:"blalba"
11:32:36:124.id_0000.1:"blalba"
11:33:49:124.id_0000.1:"blalba"
11:36:23:124.id_0000.1:"blalba"
12:10:21:124.id_0000.1:"blalba"
12:20:21:124.id_0000.1:"blalba"
12:30:21:124.id_0000.1:"blalba"
1 Like

Thanks for your solution but it's work only for my example.
For example if I choose time range between 15:40:00 and 23:10:00
this solution doens't work :frowning:

Thanks a lot for this very nice solution :slight_smile:
I have a lot of work to undestand in depth your code :slight_smile:

But , anyway to do it with awk ?

#!/usr/bin/perl

@divs=qw(hour min sec milis); # the names of the time parts I'm grabbing
@start{@divs} = $ARGV[0]=~/(\d+)/g; # throw the first four digit groups into a hash naming them by the keys in @div
@end{@divs} = $ARGV[1]=~/(\d+)/g; # As above but the second argument to the script
#open (DATA, '<', "$ARGV[2]"); # if you were supplying a logfile as an argument to the command
while(<DATA>){ 
    @time{@divs} = $_ =~/(\d+):/g; # as above, but for each line of the log file.
         # the record time is after start time and not after end time for the window we are looking for.
    if ((after(\%time, \%start)) && (! after(\%time, \%end))){ # see below for how after works the "\%" passes references to the hashes
        print ; # print out the line 
    }
}
sub after{ # boolean check to see if a time is after a given limit
    my ($time,$limit)=@_; # the arguments supplied
    for (@divs){ # go through the keys in order
        if ($time->{$_} < $limit->{$_}){ 
            return 0; # if the first non-equal division (hour min sec milli) is less than the limit then this is not after the time
        }
        elsif ($time->{$_} > $limit->{$_}){
            return 1; # the other case
        }
    }
    return 1; # millisecond equivalent, display it
}
__DATA__
10:02:23:124.id_0000.1:"blalba"
10:05:26:124.id_0000.1:"blalba"
10:10:32:124.id_0000.1:"blalba"
11:32:36:124.id_0000.1:"blalba"
11:33:49:124.id_0000.1:"blalba"
11:36:23:124.id_0000.1:"blalba"
12:10:21:124.id_0000.1:"blalba"
12:20:21:124.id_0000.1:"blalba"
12:30:21:124.id_0000.1:"blalba"

The above would be called as

~/$ getLogWindow.pl 11:30:00 11:40:00 application.log

If you used the open command to access the log file in ARGV[2]

I'm not sure that an awk command will give you a neater solution, however I'm frequently amazed at what some people manage to get awk to do on this site, though I do wonder if they can see what and how it is doing 6 months later :wink:

This will help you...

nawk -F':' -v v1="11:30:00" -v v2="11:40:00" 'OFS=":"{v=$1":"$2":"$3}{if(v>v1 && v<v2)print}' inputfile
11:32:36:124.id_0000.1:"blalba"
11:33:49:124.id_0000.1:"blalba"
11:36:23:124.id_0000.1:"blalba"

Thanks
Sha

1 Like

Oh tanks again for all this explanation... it is very helpful!

That's why I love awk so much... simple, short and perfect!
Thanks a lot :p:b: