Displaying log file pattern output in tabular form output

Shell Programming and Scripting
1

Hi All,
I have result log file which looks like this (below): from the content need to consolidate the result and put it in tabular form

1). Intercomponents Checking
Passed: All Server are passed.
======================================================================
2). OS version Checking
Passed: All Server are passed.
======================================================================
3). Component registered and running check
Failed:VerifyComponent:D01A : Some Component Not RUNNING. ATS 0/1 :
Failed:VerifyComponent:C01A : Some Component Not RUNNING. MTD 0/1 :
======================================================================
4). Verify Server Connection, for ATS/MTD/IDT components
Failed:ConnectionCheck:D01A : Gateway fail to connect to SAS server!
Failed:ConnectionCheck:B01A : Gateway fail to connect to SCAD server: u-!
Failed:ConnectionCheck:A01A : Gateway fail to connect to CSEH server
======================================================================
5). Verify Service Availability, for Z2ZS or ZDS/ZDZ or IDX/MDX or ALRX
Failed:VerifyServiceAvailability:D01A: Service IDD is NOT up.
Failed:VerifyServiceAvailability:D01A: Service DD is NOT up.
======================================================================
6). Verify Component Resiliency
Failed:VerifyResiliency:SH01: Service IDD(45002): Only HH07A(274) provide the service IDD. Please check HH07B(275)
Failed:VerifyResiliency:BH01: Service EL: No service provided.
======================================================================
7). Verify RMCmon component
Passed: All Server are passed.
======================================================================
8). Checking the Core file
Failed:ddFileGenerated:CD01A : 1 : 1485297 Jan 7 11:11

i tried to use some grep command to print the output file content in tabular form

grep '^[0-9]).' outputfilename
grep -A 1 -B 1 '^[0-9]).' outputfilename // this command didn't work since it's not GNU version

tabluar form should look like :

|========================================|
| Consolidated Report Status |
|========================================|
|SI No CHECKS STATUS |
|========================================|
|1)Intercomponents Checking : Passed |
|----------------------------------------------------|
|2)OS version Checking : Passed |
|----------------------------------------------------|
|3)Component registered : Failed |
|----------------------------------------------------|
|4)Verify Server Connection : Failed |
|----------------------------------------------------|
|5)Verify Service Availability : Failed |
|----------------------------------------------------|
|6)Verify Component Resiliency : Failed |
|----------------------------------------------------|
|7)Verify RMCmon component : Passed |
|----------------------------------------------------|
|8)Checking the Core file : Failed |
|========================================|
| Total Passed : 2 |
| Total Failed : 5 |
| Total executed: 7 |
|========================================|
2 
awk 'BEGIN {
        print "|========================================|"
        print "|        Consolidated Report Status       |"
        print "|========================================|"
        print "|SI No   CHECKS          STATUS           |"
        print "|========================================|"
 } /^[0-9]/ {
        c++;
        print "|------------------------------------------|"
        printf "| %s ",$0; getline; s=$0; sub(/:.*/,"",s); (s=="Passed")?++p:++f; printf " : %s |\n", s;
        print "|------------------------------------------|"
 } END {
        print "|========================================|"
        print "| Total Passed:" p "                      |"
        print "| Total Failed:" f "                      |"
        print "| Total Executed:" c "                    |"
        print "|========================================|"
}' filename
1 Like
3

Thanks for the reply bipinajith.

but when i try to execute the code it's report error at line number 10 ie,

root@OPT# ./output.sh
awk: syntax error near line 10
awk: illegal statement near line 10
awk: syntax error near line 10
awk: illegal statement near line 10

Error line is :

printf "| %s ",$0; getline; s=$0; sub(/:.*/,"",s); (s=="Passed")?++p:++f; printf " : %s |\n", s;

not able to figure it out what's wrong in the above line.

Apart from that,

I have another question, from the above line i can see you are comparing "s" with Passed, how about "Failed" . does it print Passed or Failed string. If i want to display "No Result" if it's not Passed or Failed then how to incoporate that in the above line.

Will it be possible to add the color code for "Passed"/"Failed"/"No Result" as green/red/blue in the tabular output ?

4

Use nawk instead on Solaris or SunOS

Here is modified code with "No Result" added:

nawk 'BEGIN {
        print "|========================================|"
        print "|        Consolidated Report Status       |"
        print "|========================================|"
        print "|SI No   CHECKS          STATUS           |"
        print "|========================================|"
 } /^[0-9]/ {
        c++;
        print "|------------------------------------------|"
        printf "| %s ",$0; getline; s=$0; sub(/:.*/,"",s);
        if(s=="Passed") ++p; if(s=="Failed") ++f; if(s=="") ++n;
        printf " : %s |\n", s;
        print "|------------------------------------------|"
 } END {
        print "|========================================|"
        print "| Total Passed: " p "                      |"
        print "| Total Failed: " f "                      |"
        print "| Total No Result: " n "                 |"
        print "| Total Executed: " c "                    |"
        print "|========================================|"
}' filename

For color code have a look at this link use printf instead of echo

5

Thanks for the reply. nawk works but there is some issue with passed/failed/no result count.

In the below output, if you see there is 3 Passed, 5 Failed and 1 No result (number 9) but not sure why it showing wrong information.

root@OPT # ./output.sh
|========================================|
|        Consolidated Report Status      |
|========================================|
|   SI No     CHECKS           STATUS    |
|========================================|
|------------------------------------------|
| 1). Interface Checking  : Passed   |
|------------------------------------------|
|------------------------------------------|
| 2). OS version Checking  : Passed   |
|------------------------------------------|
|------------------------------------------|
| 3). Comp running check  : Failed   |
|------------------------------------------|
|------------------------------------------|
| 4). Verify Server Connection, for ADD/ZDZ/IDD components  :           Failed   |
|------------------------------------------|
|------------------------------------------|
| 5). Verify Service Availability, for PZPZS or ADD/ZDZ or IDD/DDM or ALRZX  : Failed   |
|------------------------------------------|
|------------------------------------------|
| 6). Verify Resiliency  : Failed   |
|------------------------------------------|
|------------------------------------------|
| 7). Verify mon component  : Passed   |
|------------------------------------------|
|------------------------------------------|
| 8). Checking the Core file  :  Failed   |
|------------------------------------------|
|------------------------------------------|
| 9). Data request checking and Usage collection checking  : Please execute /tmp/TestPerm.bash script   |
|------------------------------------------|
|========================================|
| Total e[1;32;40m Passed: 3         |
| Total e[1;31;40m Failed: 3           |
| Total No Result:          |
| Total Executed: 9                      |
|========================================|

Many Thanks for the color code link, i tried that too but again it doesn't evalutes the value instead it just prints. am using printf but both below line didn't work

printf "| Total \e[1;32;40m Passed:\ " p "\        |\n"
print  "| Total \e[1;31;40m Failed: " f "           |"
6

Apply this modification:

if(s=="Passed") ++p; if(s=="Failed") ++f; if((s!="Passed")&&(s!="Failed)) ++n;
7

Thanks bipinajith.

I add line to handle conditions properly but somehow it takes the No Result count as 3 though it should be 1 (point 9, this should be in No Result count since there is no Passed/Failed string).

more over, for Failed count, it shows 3 instead of 5 this is where am getting confused. why is it missing to count another 2 as Failed status. Is that because of (point 4 and 5 , which is out of line)

if(s=="Passed") ++p; if(s=="Failed") ++f; if((s!="Passed")&& (s!="Failed")||(s=="") ) ++n;

ouput we get now is :

root@OPT # ./output.sh
|========================================|
|        Consolidated Report Status      |
|========================================|
|   SI No     CHECKS           STATUS    |
|========================================|
|------------------------------------------|
| 1). Interface Checking  : Passed   |
|------------------------------------------|
|------------------------------------------|
| 2). OS version Checking  : Passed   |
|------------------------------------------|
|------------------------------------------|
| 3). Comp running check  : Failed   |
|------------------------------------------|
|------------------------------------------|
| 4). Verify Server Connection, for ADD/ZDZ/IDD components  :           Failed   |
|------------------------------------------|
|------------------------------------------|
| 5). Verify Service Availability, for PZPZS or ADD/ZDZ or IDD/DDM or ALRZX  : Failed   |
|------------------------------------------|
|------------------------------------------|
| 6). Verify Resiliency  : Failed   |
|------------------------------------------|
|------------------------------------------|
| 7). Verify mon component  : Passed   |
|------------------------------------------|
|------------------------------------------|
| 8). Checking the Core file  :  Failed   |
|------------------------------------------|
|------------------------------------------|
| 9). Data request checking and Usage collection checking  : Please execute /tmp/TestPerm.bash script   |
|------------------------------------------|
|========================================|
| Total e[1;32;40m Passed: 3         |
| Total e[1;31;40m Failed: 3           |
| Total No Result:    3                   |
| Total Executed: 9                      |
|========================================|

---------- Post updated 01-02-13 at 02:59 AM ---------- Previous update was 31-01-13 at 03:40 AM ----------

Hi bipinajith/All,
Can you please explain the below line of code, I will try to sort out other issues

printf "| %s ",$0; getline; s=$0; sub(/:.*/,"",s);
8 
printf "| %s ",$0;      Print current input record
getline;                Read next input record
s=$0;                   Assign s = entire record
sub(/:.*/,"",s);        Remove everything followed by colon : from s variable value
9

Thanks bipinajith for detail explination.
I found out the exact reason why it was giving wrong results count.
In the logfile, some of the lines starts with single space and then string either Passed/Failed
something like this input file :

1). Intercomponents Checking
Passed: All Server are passed.
======================================================================
2). OS version Checking
 Passed: All Server are passed.
======================================================================
3). Component registered and running check
 Failed:VerifyComponent:D01A : Some Component Not RUNNING. ATS 0/1 :
Failed:VerifyComponent:C01A : Some Component Not RUNNING. MTD 0/1 :
======================================================================
4). Verify Server Connection, for ATS/MTD/IDT components
 Failed:ConnectionCheck:D01A : Gateway fail to connect to SAS server!

from the above logfile, we expect result count as
Passed: 2
Failed: 2
but after we run the script, it gives result count as :
Passed: 1
Failed: 1
Is there way we can Trim space before reading the line ?
In meantime, i tried the text color logic from the url which you said. Problem is we can't print the text color inside
the nawk command.
so I thought i will print those count statements after the nawk command but i don't know how to pass the variable
value from nawk command.
Can you please advice on it.

tmp_p=0
tmp_f=0
tmp_n=0
tmp_c=0
nawk 'BEGIN {
        print "|========================================|"
        print "|        Consolidated Report Status      |"
        print "|========================================|"
        print "|   SI No     CHECKS           STATUS    |"
        print "|========================================|"
 } /^[0-9]/ {
        c++;
        print "|------------------------------------------|"
        printf "| %s ",$0; getline; s=$0; sub(/:.*/,"",s);
        if(s=="Passed") ++p; if(s=="Failed") ++f; if((s!="Passed")&& (s!="Failed")||(s==" ") ) ++n;
        tmp_p=p
        tmp_f=f
        tmp_n=n
       #if(s=="Passed") ++p; if(s=="Failed") ++f; if(s==" ") ++n;
        printf " : %s   |\n", s;
        print "|------------------------------------------|"
 } END {
        print "|========================================|"
        printf "| Total Passed:" p "                    |\n"
        print  "| Total Failed: " f "                   |"
        print "| Total No Result: " n "                 |"
        print "| Total Executed: " c "                      |"
        print "|========================================|"
}' $1
        printf "|\e[1;37;40m========================================|\n"
        printf "| Total \e[1;32;40m Passed: $tmp_p                    |\n"
        printf  "| Total \e[1;31;40m Failed:  $tmp_f                   |\n"
        printf "| Total \e[1;34;40m No Result:  $tmp_n                 |\n"
        printf "| Total \e[1;35;40m Executed:  $c                   |\n"
        printf "|\e[1;37;40m========================================|\n"
10

ok, i correct the Trim part by updating this line :

printf "| %s ",$0; getline; s=$0;sub(" ","", s);sub(/:.*/,"",s);

Now only leftout is how to pass the nawk variable values outside the loop.