Script to extract fields

Shell Programming and Scripting
1

Dear all,

please see the logs shown below:

12-12 08:47:37.545 DBG AGIML SERVER[1]..............................write() (<agiml><header><responsetype>TOPUP</responsetype></header><response><auditfields/><resultcode>999</resultcode>
<account>9773575450</account><transno>1317</transno><incode>BPL</incode><resultdescription></resultdescription><inresultcode>1</inresultcode><inmtransno>229051853686229</inmtransno><flowControlCode></flowControlCode><flowControlLevel></flowControlLevel><intransno></intransno><acls/><authorities/><records/></response></agiml>)

Now i want to write a script which will extract the fields like this

TOPUP,9773575450,1317,BPL,1

these are the fields shown in the different tags.

And one thing more the fields between the tag <agiml>............</agiml>,
it is taking it a single column of log.

example less file | grep "9773575450"
will display the whole line.

2

Are those fields always the same you need?

Btw, better use

grep "9773575450" file
# instead of
less file | grep "9773575450"

since the less is needless.

3 
sed '
s/.*<agiml>//  # remove up to <agiml>
s|</agiml>.*|| # remove </agiml> to end of line
s/<[^>]*>/,/g  # convert tags to commas
s/,,*/,/g      # compress multiple commas to a single comma
s/^,//         # remove leading comma
s/,$//         # remove trailing comma
' FILE
4

Thanks,
It works, but there is a little problem
the output is like this

999,9821376954,1143,BPL
,228889066557802

999,9773575450,1317,BPL
,229051853686229
but i want the second line should come in the first line followed by BPL

Best Regards,
Akhtar Bhat

5

So "999" is wanted too? In your first post it is not wanted and TOPUP was wanted too. Please give a statement what you want as output.

Edit:

Just in case you want all values between > and <, here you go:

awk '$1=$1' RS="<[^>]*>" infile2| sed -e '1d; $d'| sed -e :a -e 'N; s/\n/,/; ba'
TOPUP,999,9773575450,1317,BPL,1,229051853686229

Tried cfajohnson's - it worked flawless with my sed too.

6

Dear Sir,
Thanks for the help.
The reply you made is good but it is not working properly example when new logs are coming in the log file, It is shown below

Example logs:-

10-12 11:34:30.452 DBG AGIML SERVER[1]..............................write() (<agiml><header><responsetype>TOPUP</responsetype></header><response><auditfields
/><resultcode>999</resultcode><account>9821376954</account><transno>1143</transno><incode>BPL</incode><resultdescription></resultdescription><inresultcode>1<
/inresultcode><inmtransno>228889066557802</inmtransno><flowControlCode></flowControlCode><flowControlLevel></flowControlLevel><intransno></intransno><acls/><
authorities/><records/></response></agiml>)
12-12 08:47:37.545 DBG AGIML SERVER[1]..............................write() (<agiml><header><responsetype>TOPUP</responsetype></header><response><auditfields
/><resultcode>999</resultcode><account>9773575450</account><transno>1317</transno><incode>BPL</incode><resultdescription></resultdescription><inresultcode>1<
/inresultcode><inmtransno>229051853686229</inmtransno><flowControlCode></flowControlCode><flowControlLevel></flowControlLevel><intransno></intransno><acls/><
authorities/><records/></response></agiml>)

Output of your command :-

TOPUP,999,9821376954,1143,BPL,1,228889066557802,) 12-12 08:47:37.545 DBG AGIML SERVER[1]..............................write() (,TOPUP,999,9773575450,1317,BPL,1,229051853686229

The output should be like this:-
TOPUP,999,9821376954,1143,BPL,1,228889066557802
TOPUP,999,9773575450,1317,BPL,1,229051853686229

Best Regards,
Akhtar Bhat

7

cfajohnson's works flawless with me. Maybe try another version of sed.

8

Nvm, another similar approach, that might work on your system:

awk '$1=$1' FS="<[^>]*>" infile| cut -d"(" -f3| sed 's/[[:space:]]\{1,\}/,/g; s/^,//; s/,)$//'
TOPUP,999,9821376954,1143,BPL,1,228889066557802
TOPUP,999,9773575450,1317,BPL,1,229051853686229