Count of matched pattern occurences by time in a log file

Shell Programming and Scripting
1

We have a log file, the format is similar to this:

08/04/2011 05:03:08 Connection Success
08/04/2011 05:13:18 Connection Success
08/04/2011 05:23:28 Connection Fail
08/04/2011 05:33:38 Connection Success
08/04/2011 06:14:18 Connection Success
08/04/2011 06:24:28 Connection Fail
08/04/2011 06:34:38 Connection Fail
08/04/2011 07:14:48 Connection Success
08/04/2011 07:25:18 Connection Success
08/04/2011 07:35:28 Connection Fail

I am hoping to write a shell script which can parse the file and have the output like:

08/04/2011 between 05:01 to 06:00 Success: 2 Fail: 1
08/04/2011 between 06:01 to 07:00 Success: 1 Fail: 2
08/04/2011 between 07:01 to 08:00 Success: 2 Fail: 0

Thanks very much for your time and help.

2

can you give more details and the desired output you are expecting
If possible please also paste your current shell script

3

Hi Harris,

I have posted my desired output as:

08/04/2011 between 05:01 to 06:00 Success: 2 Fail: 1
08/04/2011 between 06:01 to 07:00 Success: 1 Fail: 2
08/04/2011 between 07:01 to 08:00 Success: 2 Fail: 0

Thanks

4 
awk -F ":" '/Success/{s[$1]++;}/Fail/{f[$1]++}
END {for (i in s) {split(i,a," "); printf "%s Between %02d:00 to %02d:59 Success: %d Fail: %d\n", a[1],a[2],a[2],s,f}}' infile

08/04/2011 Between 05:00 to 05:59 Success: 3 Fail: 1
08/04/2011 Between 06:00 to 06:59 Success: 1 Fail: 2
08/04/2011 Between 07:00 to 07:59 Success: 2 Fail: 1
5

Perl

#!/usr/bin/perl

use Time::Local;

while(<DATA>) {
chomp;
@dt=unpack("A2 x1 A2 x1 A4 x1 A2 x1 A2 x1 A2",$_);
if ($dt[3] ne $cur_hr) {
        print substr($start_dt,1,10),"Between " ,substr($start_dt,11) ," and " , substr($end_dt,11)," Success ", $suc ," and Fail ", $fail,"\n" if ($suc || $fail);
        $start_dt=$dt[0]."/".$dt[1]."/".$dt[2]." ".$dt[3].":00:00";
        $mon=$dt[1] - 1;
        $yr= $dt[2] - 1900;
        $sec_start=timelocal(0,0,$dt[3],$dt[0],$mon,$yr);

        $sec_end=$sec_start + 3600;
        $end_dt=$dt[0]."/".$dt[1]."/".$dt[2]." ".$dt[3].":59:00";
        $suc=0;$fail=0;

}
        $cur_sec=timelocal($dt[5],$dt[4],$dt[3],$dt[0],$dt[1]-1,$dt[2]-1900);
        if ( $cur_sec >= $sec_start && $cur_sec <= $sec_end) {
                if ( /\bSuccess\b/ ) {++$suc;}else{++$fail;}
                }
$cur_hr=$dt[3];
}

print substr($start_dt,1,10),"Between " ,substr($start_dt,11) ," and " , substr($end_dt,11)," Success ", $suc ," and Fail ", $fail,"\n" if ($suc || $fail);
__DATA__
08/04/2011 05:03:08 Connection Success
08/04/2011 05:13:18 Connection Success
08/04/2011 05:23:28 Connection Fail
08/04/2011 05:33:38 Connection Success
08/04/2011 06:14:18 Connection Success
08/04/2011 06:24:28 Connection Fail
08/04/2011 06:34:38 Connection Fail
08/04/2011 07:14:48 Connection Success
08/04/2011 07:25:18 Connection Success
08/04/2011 07:35:28 Connection Fail
6

Ruby(1.9+)

h=Hash.new(0)
File.open("file").each do |line|
    date,time,conn,status = line.split
    str = sprintf("%s|%s|%s", date,time[0,2],status)
    h[ str ] += 1
end
final={}
h.each do|k,v|
    s =  k.split("|")
    (final[ s[0,2].join("|") ] ||= [] ) << s[2]+":"+v.to_s
end
final.each{|x,y| a=x.split("|") ; puts "#{a[0]} #{a[1]}:00-#{a[1]}:59 : #{y}" }

Experimentation

$ ruby test.rb
08/04/2011 05:00-05:59 : ["Success:3", "Fail:1"]
08/04/2011 06:00-06:59 : ["Success:1", "Fail:2"]
08/04/2011 07:00-07:59 : ["Success:2", "Fail:1"]
7

This is great, thanks very much for all your help.