awk/sed for parsing file

Shell Programming and Scripting
1

Hi All,

I have a log file like this

E Mon Oct 06 00:17:08 2008  xxx2  cm:10614  fm_pi2_svc_iptv_purchase.c:149 1:pin_deferred_act:10601:11:169:1223245028:16
        pi2_op_svc_iptv_purchase error
        <location=PIN_ERRLOC_FM:5 class=PIN_ERRCLASS_SYSTEM_DETERMINATE:1 errno=PIN_ERR_NOT_FOUND:3>
D Mon Oct 06 00:42:26 2008  xxx2  cm:10611  fm_pi2_mov_wal_change.c:341 1:xxxxapp2:pin_deferred_act:10601:2:169:1223246545:104
        WAL_PRODUCT_CHANGE input flist
D Mon Oct 06 00:42:26 2008  xxxxapp2  cm:10611  fm_pi2_mov_wal_change.c:341 1:xxxxapp2:pin_deferred_act:10601:2:169:1223246545:104
        WAL_PRODUCT_CHANGE input flist
E Mon Oct 06 00:17:08 2008  xxxxapp2  cm:10614  fm_pi2_svc_iptv_purchase.c:149 1:xxxxapp2:pin_deferred_act:10601:11:169:1223245028:16
        pi2_op_svc_iptv_purchase error
        <location=PIN_ERRLOC_FM:5 class=PIN_ERRCLASS_SYSTEM_DETERMINATE:1 errno=PIN_ERR_NOT_FOUND:3>
        <location=PIN_ERRLOC_FM:5 class=PIN_ERRCLASS_SYSTEM_DETERMINATE:1 errno=PIN_ERR_NOT_FOUND:3>
        <location=PIN_ERRLOC_FM:5 class=PIN_ERRCLASS_SYSTEM_DETERMINATE:1 errno=PIN_ERR_NOT_FOUND:3>
D Mon Oct 06 00:17:08 2008  xxxxapp2  cm:10614  fm_pi2_svc_iptv_purchase.c:149 1:xxxxapp2:pin_deferred_act:10601:11:169:1223245028:16
	XXXXXXXXXXXXX

I need to filter this log file based on the process id which is marked as bold.
if i am giving 10614 as a input parameter it should show only the logs of that process id like this :

E Mon Oct 06 00:17:08 2008  xxx2  cm:10614  fm_pi2_svc_iptv_purchase.c:149 1:pin_deferred_act:10601:11:169:1223245028:16
        pi2_op_svc_iptv_purchase error
        <location=PIN_ERRLOC_FM:5 class=PIN_ERRCLASS_SYSTEM_DETERMINATE:1 errno=PIN_ERR_NOT_FOUND:3>
E Mon Oct 06 00:17:08 2008  xxxxapp2  cm:10614  fm_pi2_svc_iptv_purchase.c:149 1:xxxxapp2:pin_deferred_act:10601:11:169:1223245028:16
        pi2_op_svc_iptv_purchase error
        <location=PIN_ERRLOC_FM:5 class=PIN_ERRCLASS_SYSTEM_DETERMINATE:1 errno=PIN_ERR_NOT_FOUND:3>
        <location=PIN_ERRLOC_FM:5 class=PIN_ERRCLASS_SYSTEM_DETERMINATE:1 errno=PIN_ERR_NOT_FOUND:3>
        <location=PIN_ERRLOC_FM:5 class=PIN_ERRCLASS_SYSTEM_DETERMINATE:1 errno=PIN_ERR_NOT_FOUND:3>
D Mon Oct 06 00:17:08 2008  xxxxapp2  cm:10614  fm_pi2_svc_iptv_purchase.c:149 1:xxxxapp2:pin_deferred_act:10601:11:169:1223245028:16
	XXXXXXXXXXXXX

How can i do this through awk or sed ?

Thanks in advance

2

Use nawk or /usr/xpg4/bin/awk on Solaris.

awk 'END {
  if (f) print r
  }
/ Mon / {
  if (f) print r
  r = f = 0
  }
$0 ~ "cm:" id { f++ }  
{ r = r ? r RS $0 : $0 }
' id=<id> logfile
3

Hi,
Thanks for the reply...
I tried like this .

awk 'END {
if (f) print r
}
/ Mon / {
if (f) print r
r = f = 0
}
$0 ~ "cm:" id { f++ }
{ r = r ? r RS $0 : $0 }
' id=10614 logfile.txt

But i am getting this error ;

./awk.sh
awk: syntax error near line 4
awk: bailing out near line 4

The log file not always start with Mon, it can be any other day like Wed,Fri etc ..

Thanks and Regards,
Subin

4

Did you follow the nawk/XPG awk suggestion?

OK,
change the regex from / Mon / to !/[1]/.

  1. \t ↩︎

5

How can i use "nawk/XPG awk suggestion" in my code ?

if i am trying following command :

which awk - /usr/bin/awk is the output
which nawk - /usr/bin/nawk is the output

6

Just run nawk or /usr/xpg4/bin/awk instead of awk.

7

Hi,

I used nawk and its working fine.

Thanks alot Radoulov for ur constant support :slight_smile:

Thanks and Regards,
Subin

8

And can u pls expain simply how code is working ??

Thanks in advance

9

Yes.
It's an AWK script:

awk '...' var1=value [var2=value ... varn=value] inputfile(s)

var=value assigns a value to a variable var accessible inside the AWK code.
So id=123 is the desired id to be passed to the program.

Following the code logic we have:

  1. Construct the logical record (r) by concatenating all the records seen so far:

record r -> if record is not empty: r ? -> add a record separator (RS, newline by default) and the current record ($0): r RS $0, else (record is empty, it's the first access -> assign the value of the current record: : $0
This is the meaning of the following expression:

{ r = r ? r RS $0 : $0 }
  1. Check if the current record matches the pattern "cm:" followed by the value of the variable id (see above): $0 ~ "cm:" id. If the test returns true, auto increment the value of the variable f (f for flag, marker): { f++ }.
$0 ~ "cm:" id { f++ }
  1. If the current record does not match the pattern [1] : the line does not begin with a blank character (tab or space), these are your E, D etc records, do the following:
  • check if the value of the variable f in Boolean context returns true (is not an empty string or has a numeric value 0): if it's true (not 0, see 2. above), this logical record contains our id, so we print it: print r.
  • reset the r and the f variables, we will initialize them after if needed.
!/^[\t ]/ {
  if (f) print r
  r = f = 0
  }
  1. After reading the entire input check if we have something to print.
    This is because of the build (r) -> set (f) -> check after (!/[2]/) logic:
    we print the previous when we reach the current. So without the END block we may miss the last one.
END {
  if (f) print r
  }

Hope this helps.

  1. \t ↩︎

  2. \t ↩︎

10

Thnks alot radoulov for ur detailed explanation

11

Thanks for the explanation