Shell script to parsing log

Hi I Have log like this :
0 234: [APPLICATION 1] {
3 2: [sequenceNumber] 04 EE
7 14: [timeStamp] '20081114081'
23 1: [trigger] 00
79 10: [msisdn] '38809'
91 15: [imsi] '528111510010159'
143 29: [applicationDescription] 'Streaming/downloading service'
174 3: [custom0] 'MTV'
179 43: [contentUrl] 'rtsp://172.28/MTV2GO-Loop.sdp'
224 1: [contentRate] 05
227 1: [priceType] 01
230 2: [currencyCode] 39 36
234 1: [currencyExponent] FE
: }
237 248: [APPLICATION 1] {
240 2: [sequenceNumber] 04 EF
244 14: [timeStamp] '20081114082116'
260 1: [trigger] 02
316 10: [msisdn] '38809'
328 15: [imsi] '528111510010'
360 1: [paymentMethod] 01
380 29: [applicationDescription] 'Streaming/downloading service'
411 3: [custom0] 'MTV'
416 43: [contentUrl] 'rtsp://172.28/MTV2GO-Loop.sdp'
461 3: [transferredVolume] 19 83 87
466 1: [timeDuration] 5D
469 1: [debitedCost] 08
472 1: [chargingStatus] 01
475 1: [contentRate] 05
478 1: [priceType] 01
481 2: [currencyCode] 39 36
485 1: [currencyExponent] FE
: }
712 236: [APPLICATION 1] {
715 2: [sequenceNumber] 04 F1
719 14: [timeStamp] '20081114082'
735 1: [trigger] 01
791 10: [msisdn] '38997'
803 15: [imsi] '528111500081770'
820 10: [ipAddress] '10.84.8.66'
832 1: [clientProtocol] 01
835 1: [paymentMethod] 01
855 29: [applicationDescription] 'Streaming/downloading service'
886 35: [contentUrl] 'rtsp://172.28/cnn.sdp'
923 3: [transferredVolume] 26 D0 CD
928 2: [timeDuration] 00 A9
932 1: [debitedCost] 0E
935 1: [chargingStatus] 01
938 1: [contentRate] 05
941 1: [priceType] 01
944 2: [currencyCode] 39 36
948 1: [currencyExponent] FE
: }
.....
.....

I Want that log become just only "[trigger] 02"
and then collect into Timestamp,msisdn,timeduration,contentUrl

anybody can help me ??

I've try with this and its not working :stuck_out_tongue:

for FILE in $filename
do
temp1=`cat $FILE | grep "timeStamp]" | awk '{print$4}' | cut -d"'" -f2`
temp2=`cat $FILE | grep "trigger] 02" | awk '{print$4}'`
temp3=`cat $FILE | grep "msisdn]" | awk '{print$4}' | cut -d"'" -f2`
temp4=`cat $FILE | grep "contentUrl]" | awk '{print$4}' | cut -d"'" -f2`
temp5=`cat $FILE | grep "timeDuration]" | awk '{print$4}'`
done

echo $temp1 $temp2 $temp3 $temp4 $temp5

This should give the desired items of the section [trigger] 2. It prints the 3th and the 4th field, adjust and play around with the code if you only want to print the 4th field.

awk '
/\[trigger\] 02/{print s;f=1}
f && /\[msisdn\]/{print $3,$4}
f && /\[contentUrl\]/{print $3,$4}
f && /\[timeDuration\]/{print $3,$4;exit}
{s=$3 " " $4}' logfile

Explanation:

/\[trigger\] 02/{print s;f=1}

if we match [trigger] 2, print the time stamp we've saved in the variable s (1 line above) and assign 1 (true) to the variable f.

f && /\[msisdn\]/{print $3,$4}
f && /\[contentUrl\]/{print $3,$4}
f && /\[timeDuration\]/{print $3,$4;exit}

If f is true (1) print field 3 and 4 if one of the line match. After printing the time duration exit the program.

{s=$3 " " $4}' logfile

Store the actual line in the variable s, this is 1 line before the [trigger] line.

Use nawk or /usr/xpg4/bin/awk on Solaris.

Regards

Yep! it's working..Thanks Franklin52

Result :

But the problem is the log bigger.. contain the format like that

[APPLICATION 1] { ----> begin

;;;;;;

} ----> end

it mixed from [trigger] 00, [trigger] 01 , and [trigger] 02
and the information from [trigger] 01 there are always no value of [timeDuration]

Or should i give the example

Any solution ??

The problem is not clear to me, can you give an example?

Regards

Maybe you can see the log here

and i want that log become just only "[trigger] 02"
and then collect into Timestamp,msisdn,timeduration,contentUrl

the log always begin
[APPLICATION 1] {

and end with }

should i separate per index and then

to make it clearly you can see here

i still want that log become just only "[trigger] 02"
and then collect into Timestamp,msisdn,timeduration,contentUrl

If you see the stream always begin with :
[APPLICATION 1] { ----> begin

;;;;;;

} ----> end

Should i separate into index then,,

Regards, :slight_smile:

Do you want the output of all the sections with "[trigger] 02"?
Post the desired output.

yep,, the output is some of query that "[trigger] 2" have

The output will be like this :

timeStamp | msisdn | contentUrl | timeDuration |
20081114082116 |6738809966|rtsp://172.28.13.66/mtvloop/MTV2GO-Loop.sdp| 5D
20081114082841 |6738621251|rtsp://172.28.13.66/mtvloop/RIA-Loop.sdp|13
....

This should give you the desired output:

awk '
/\[trigger\] 02/{f=1}
/\[timeStamp\]/{ts=$4}
f && /\[msisdn\]/{msi=$4}
f && /\[contentUrl\]/{cU=$4}
f && /\[timeDuration\]/{tD=$4}
f && /APPLICATION/{
  s= ts "|" msi "|" cU "|" tD
  gsub("\047","",s)
  print s
  f=0
}' logfile

Regards

yah.. success.. it works
I've learn the script and the tutorial that u give me..
but i'm still confuse about that..

can u explain the script.

thanks..

Ok, here we go:

awk '
/\[trigger\] 02/{f=1}
/\[timeStamp\]/{ts=$4}
f && /\[msisdn\]/{msi=$4}
f && /\[contentUrl\]/{cU=$4}
f && /\[timeDuration\]/{tD=$4}
f && /APPLICATION/{
  s= ts "|" msi "|" cU "|" tD
  gsub("\047","",s)
  print s
  f=0
}' logfile

Explanation:

/\[trigger\] 02/{f=1}

Set the flag f if the a line matches the pattern [trigger] 02.

/\[timeStamp\]/{ts=$4}
f && /\[msisdn\]/{msi=$4}
f && /\[contentUrl\]/{cU=$4}
f && /\[timeDuration\]/{tD=$4} 

If the flag is set and the line matches the pattern, assign the 4th field to the variables msi, cU and tD.
The timestamp is on a line before the pattern "[trigger] 02", thus the variable for the timestamp ts must be filled unconditional.

f && /APPLICATION/{
  s= ts "|" msi "|" cU "|" tD
  gsub("\047","",s)

If the flag is set and the line matches the pattern "APPLICATION", use the string s to format the variables and to delete the quotes.

  print s
  f=0

Print the string s and unset the flag f.

Regards