Interesting awk/Perl/sed parsing challenge

UNIX for Advanced & Expert Users
1

I have a log with entries like:

out/target/product/imx53_smd/obj/STATIC_LIBRARIES/libwebcore_intermediates/Source/WebCore/bindings/V8HTMLVideoElement.cpp
[22:14:58]: target thumb C++: libwebcore <= out/target/product/imx53_smd/obj/STATIC_LIBRARIES/libwebcore_intermediates/Source/WebCore/bindings/V8ImageData.cpp
[22:14:59]: target thumb C++: libwebcore <=

If you notice the time stamps between the brackets.. what I need to do is parse this whole log (tens of thousands of lines) for the biggest gaps between any two consecutive time stamps. The time spans about 3.5 hours so the minutes column will be reset back to 0. I just want to know where the biggest delays occur in this build.

2

What have you tried so far? Did you search these forums for starting points?

3

If you really must see my attempt, I tried it first in bash, but I have been working all day and am already a little burnt.

#!/bin/bash
set count=0
awk -F':' '{print $2}' inputfile|grep -Ev '[a-zA-Z]' |grep -v 00|
while read line
do 
  [ -n $last_line ] && echo $(( line - last_line ))
    if [[ ! $count == "0" ]]; then 
        set last_line=${line}
    fi
set count +=1
done
4

Not sure if your awk will allow for a regex as field separator, but give this a shot:

awk -F"[][:]"   '               {TIME = $2*3600 + $3*60 + $4}
                 NR > 2         {DELTA=TIME-LAST; if (DELTA>MAX) {MAX=DELTA; LINEMAX=NR}}
                                {LAST=TIME}
                 END            {print LINEMAX, MAX}
                ' file

Will not, alas, span midnight...

5

I'm getting an error like umatched ], what are you doing with that field separator? I don't quite understand it. My awk does accept a regex -F but it didnt like yours. you have "[][:]" which gives a fatal error, but "[ ][:]" does not give an error. What does this regex mean? Are [ and ] literals? Perhaps escape them with backslash?

6

I'm telling awk to use "]", "[", ":" as field separators. You can use just ":" and then, as the first thing, remove all "[" and "]" from $0 with gsub (). You'll have to use $1 .. $3, then.

7

use $1..$3 where?

8

In lieu of $2, $3, $4.

9

try FS="(\\[)|(\\])|(:)" - some awk's are pesky.