awk unable to print array next to each other

Dear moderators/experts

I have below data as

   1631321 2020-04-06T23:59:58.256+0300  PL-15  ER: "vc_MessageInformation=SDA: Location information cannot be obtained; vc_AdditionalInfo=Reason: Unexpected Error from HLR, Message Type: Sh-Pull, User Identity Type: MSISDN; vc_Priority=LOG_ERR"                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
   1631320 2020-04-06T23:59:51.390+0300  PL-9  ER: "vc_MessageInformation=SDA: User not defined in AS; vc_AdditionalInfo=Reason: ,Message Type: Sh-Notif, User Identity Type: Public Id,  Application Server Name: tpsps01.fmc.stc.com.sa; vc_Priority=LOG_ERR"                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              
   1631246 2020-04-06T23:56:23.561+0300  PL-6  ER: "vc_MessageInformation=SDA: SLh Service could not be processed correctly; vc_AdditionalInfo=Reason: User identity was not located in any serving node, Message Type: SLh LCS-Routing-Info, User Identity Type: IMSI; vc_Priority=LOG_ERR"                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      
   1631319 2020-04-06T23:59:51.080+0300  PL-7  ER: "vc_MessageInformation=SDA: Service could not be processed correctly; vc_AdditionalInfo=Reason: Database access failure, Message Type: Sh-Update, User Identity Type: Public Id; vc_Priority=LOG_ERR"                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         
   1631188 2020-04-06T23:54:18.454+0300  PL-9  ER: "vc_MessageInformation=SDA: AVP error; vc_AdditionalInfo=Reason: Missing AVP, Result Code: DIAMETER_MISSING_AVP, AVP name: User-Name, Message Type: Sh-Pull, User Identity Type: Public Id, vc_Priority=LOG_ERR"                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             
   1631276 2020-04-06T23:58:04.476+0300  PL-4  ER: "vc_MessageInformation=SDA: Invalid Sequence Number; vc_AdditionalInfo=Reason: ,Message Type: Sh-Update, User Identity Type: Public Id,  Sequence Number: 7344; vc_Priority=LOG_ERR"                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           
   1631014 2020-04-06T23:49:30.379+0300  PL-12  ER: "vc_MessageInformation=SDA: Inconsistency between user identities; vc_AdditionalInfo=Reason: User Identity does not correspond to Private identity, Message Type: Sh-Pull, Private Id: ; vc_Priority=LOG_ERR"  

From the above print i want output as below

vc_MessageInformation                                   vc_AdditionalInfo                                                Result Code:      AVP name:            AVP name:  Message Type          Application Server Name
 Location information cannot be obtained          Unexpected Error from HLR                                                                Sh-Pull                                     
 User not defined in AS                                                                                                                    Sh-Notif             tpsps01.fmc.spp.com.sa
 SLh Service could not be processed correctly     User identity was not located in any serving node                                        SLh LCS-Routing-Info
 Service could not be processed correctly         Database access failure                                                                  Sh-Update
 AVP error                                        Missing AVP                                           DIAMETER_MISSING_AVP, User-Name    Sh-Pull                                                                                                                                   Sh-Update
 Invalid Sequence Number                                                                                                                   Sh-Pull
 Inconsistency between user identities            User Identity does not correspond to Private identity                                    Sh-P

This mean i want to store data in 6 arrays and want to print on screen one after another

I used below awk command

awk -F'[:;,"]' '/vc_MessageInformation/{print a["m"] " " a["n"] a["o"] a["p"] " " a["q"] " " a["r"];delete a}/vc_MessageInformation/{a["m"]=$6;next}/vc_AdditionalInfo/{a["n"]=$8;next}/Result Code/{if ($9=="Result Code") a["o"]=$10;else a["o"]="";next}/AVP name/{if ($11=="AVP name") a["p"]=$12;else a["p"]="";next}/Message Type/{if ($9=="Message Type") a["q"]=$10;else a["q"]=$14;next}/Application Server Name/{if ($16=="Application Server Name") a["r"]=$17;else a["r"]="";next}END{print a["m"] " " a["n"] a["o"] a["p"] " " a["q"] " " a["r"];}' SHAPP_HC_FIL

but output only print first array a["m"] whereas i want them to print in sequence a["m"]

" " a["n"] " " a["o"] " " a["p"] " " a["q"] " " a["r"]

 

Location information cannot be obtained
 User not defined in AS
 SLh Service could not be processed correctly
 Service could not be processed correctly
 AVP error
 Invalid Sequence Number
 Inconsistency between user identities

Thanks

No surprise: the next statement after every single match stops processing the actual input line; you CANNOT print more that one item.

Dear RudC,

Thnaks after removing next.. I got the desired output.

awk -F'[;,"]' '/vc_MessageInformation/{print a["m"] "," a["n"] "," a["o"] ",";delete a}/vc_MessageInformation/{a["m"]=$2}/Message Type/{if ($4 ~ "Message Type") a["n"]=$4;{if ($6 ~ "Message Type") a["n"]=$6}}/Message Type/{if ($6 ~ "User Id") a["o"]=$6 {if ($8 ~ "User Id") a["o"]=$8;{if($5 ~ "User Identity :") a["o"]=$5}}}END{print a["m"] "," a["n"] "," a["o"] ","}' SHAPP_HC_Log > SHAPP_OUTPUT

Good to hear, although I'm getting a syntax error at or near { with your above code. After correction, the output doesn't quite look like the desired one in yur post #1.

Strong recommendation: Use - and stick to - a code structuring schema to your taste, like indenting and block building, for better readability, undaerstandability, and maintainability.

How about an approach like this one (adapt output field length if necessary):

awk '
BEGIN           {HCNT = split ("vc_MessageInformation|vc_AdditionalInfo|Result Code|AVP name|Message Type|Application Server Name", COLN, "|")
                 FMT  = "%-35.35s "
                 for (i=1; i<=HCNT; i++) printf FMT, COLN
                 printf ORS
                }
                {for (i=1; i<=HCNT; i++)        {match ($0, COLN "[^;,]*")
                                                 TMP = substr ($0, RSTART, RLENGTH)
                                                 sub (/^[^:]*: /, _, TMP)
                                                 printf FMT, TMP
                                                }
                 printf ORS
                } 
' file
vc_MessageInformation               vc_AdditionalInfo                   Result Code                         AVP name                            Message Type    ...
Location information cannot be obta Unexpected Error from HLR                                                                                   Sh-Pull         ...
User not defined in AS                                                                                                                          Sh-Notif        ...
SLh Service could not be processed  User identity was not located in an                                                                         SLh LCS-Routin  ...
Service could not be processed corr Database access failure                                                                                     Sh-Update       ...  
AVP error                           Missing AVP                         DIAMETER_MISSING_AVP                User-Name                           Sh-Pull         ...      
Invalid Sequence Number                                                                                                                         Sh-Update       ...
Inconsistency between user identiti User Identity does not correspond t                                                                         Sh-Pull         ...

Thanks a lot RudC

Yes, Agreed from my version ,The achivement was partial then i used column command to realigned the column.

But this one really great.

Can you explain this code, So I can use it if similar kind of data processing needed.

regards,
Shanul

--- Post updated at 09:41 AM ---

Dear RudC,

Is it possible to insert extra column with your code which will count similar lines or similar lines in column. Like I am using seperate awk command as below.

Output as below

1. A-Z ↩︎

It is possible, yes. Try

awk '
BEGIN           {HCNT = split ("vc_MessageInformation|vc_AdditionalInfo|Result Code|AVP name|Message Type|Application Server Name", COLN, "|")
                 TMP  = split ("40 40 22 12 16 25", FLEN) 
                 FMT  = "%-*.*s "
                 for (i=1; i<=HCNT; i++) printf FMT, FLEN, FLEN, COLN
                 printf "Count" ORS
                }
                {OUT = ""
                 for (i=1; i<=HCNT; i++)        {match ($0, COLN "[^;,]*")
                                                 TMP = substr ($0, RSTART, RLENGTH)
                                                 sub (/^[^:]*: /, _, TMP)
                                                 OUT = OUT sprintf (FMT, FLEN, FLEN, TMP)
                                                }
                 T[OUT]++
                } 
END             {for (t in T) print t, T[t]
                }
' file 

It first gets the header and fields' definition into the COLS array, and the respective lengths into FLEN to be used when formatting the fields with the "dynamic" field wifth and precision specifiers in FMT . Then, the header is printed, supplemented by the "Count" field header.
In the main block, the actual line is searched, item by item, for the header keywords plus their values. The latter are extracted into a TMP variable and collected into the OUT put line. After the line is complete, a counter array is incremented.
In the END section, the counter array is printed, index (= output lines) first, then count values. Be aware that the order of the array element retrieval is undefined in awk .

Thanks a lot RudC..

This will solve all my issue for using multiple awk liner to process the output.

From above description i got the idea in general.

But if you can explain code line by line.. I mean each variable and method. It will be grateful.
As i need to use similar for processing many different data files with different headers and columns.

Thanks again for this grateful help

Dear RudC,

Can we have changes in your script according to below data..

Fine. But: You should try to get a more profound understanding.

I don't feel this is a good approach. It would be better you check each line, variable and method yourself with the help of e.g. man awk and e.g. printing out intermediate steps / variables like COLN , OUT , or TMP . If you really, really get stuck with a statement or result, post back very specifically, and people may jump in.

This may seem tedious on first sight, but will be rewarding in the long run.

Can almost certainly be done, but the input file changed profoundly: I can't tell where and how fields / values are separated; it seems all just a looong unstructured string. So, the script / logics may need to be turned upside down. On top, some fields / characters ( ; , final " ) seem missing...

Thanks RudC for valuable advice

I will use same apprach

Here I tried below code

Result is somewhat not as expected

I have changed input file as suggested by you

1. ^: ↩︎

There was no count specified in your changed request in post #8. This is as close as I get towards your before last specification:

awk -F";" '
BEGIN           {print HD = "vc_MessageInformation      vc_AdditionalInfo       Error code      Error Text      Database server"
                 HCNT = split (HD, COLN, "\t")
                }

                {sub (/^.*"/, _)
                 for (j=1; j<=NF; j++)  {FP = 0
                                         for (i=1; i<=HCNT; i++) FP += sub ("[ ]*" COLN "[^=:]*[=:]", _, $j)
                                         if (! FP) $j = _
                                        }
                }

1
' file
vc_MessageInformation    vc_AdditionalInfo    Error code    Error Text    Database server
Failure in database operation towards the back-end  Operation: Modify 20 Type or value exists - modify/add: CDC: value #0 already exists ldap://1X.2XX.1XX.2XX:3XX
Failure in database operation towards the back-end  Operation: Modify 20 Type or value exists - modify/add: CDC: value #0 already exists ldap://1X.2XX.1XX.2XX:3XX
.
.
.
 

Rg. your next change in spec, I'd propose you proceed as indicated in post #9: change spec and script in small steps, understand discrepancies in result and operation, and come back with very specific, detailed questions. "Result is somewhat not as expected" doesn't help to track down errors, neither yourself nor others trying to help. "Why can't the error code value be extracted or isolated" is way more targeted, don't you think? Yes, this is tedious, but the only way to get through.

Thanks a lot RudC

Noted , Your advise will help me for learning better skill