first field match

Shell Programming and Scripting
1

Hi All,
Thanks for your help in advance

I am parsing a log file where the first field of each line can have the same key value but not more than 3 times in a row. Varying value of that first field changes as you go through the log but it either appears 3 times or two and sometimes only once. Example
aaa ... ..... request| ( at the end of each row is "request|")
aaa ... ..... request|
aaa ... ..... request|
ccc ... ..... "
ccc ... .... "
ddd ... .... "
ddd ... .... "
ddd ... .... "
eee ... .... "
ggg ... .... "
ggg ... .... "
HHH ... .... "
kkk ... .... "
kkk ... ..... "
kkk ... .... "
As I read this log file I would like to parse lines on the first field with their matching values and append them to a separate file.. So all 3 lines for aaa, ddd and kkk go to a file, do the same for 2 matching lines and append them to a different file and the rest , single line like eee append to a third file.
so file 1 would have
aaa ... .....
aaa ... .....
aaa ... .....
ddd ... ....
ddd ... ....
ddd ... ....
kkk ... ....
kkk ... ....
kkk ... ....

file 2 would have
ccc ... ....
ccc ... ....
ggg ... ....
ggg ... ....

file 3
eee ... ....
HHH ... ....

Thanks in advance - your help much appreciated

2

Hi,

Execute the below BASH shell script and take three output files generated, output1.log, output2.log and output3.log.

Note: Because the script needs to iterate through the whole log file at least two times, execution can be much slower when you use a large file. Try convert it into PERL then.

#!/bin/bash

mylogfile=input.log
pointer=0
index=""
index_old="$(head -n 1 $mylogfile | awk '{print $1}')"

while read raw ; do
        index="$(echo "$raw" | awk '{print $1}')"
        if test "$index_old" != "$index" ; then
                count=0
                while test $count -lt $pointer ; do
                        echo "${line[$count]}" >> output${pointer}.log
                        count=$((count + 1))
                done
                pointer=0
        fi
        line[$pointer]="$raw"
        index_old="$index"
        pointer=$((pointer + 1))
done < $mylogfile

count=0
while test $count -lt $pointer ; do
        echo "${line[$count]}" >> output${pointer}.log
        count=$((count + 1))
done

happy x-mas

meharo