Piping tail to awk to parse a log file

Shell Programming and Scripting
1

Hello all,

I've got what I'm pretty sure is a simple problem, but I just can't seem to work past it. I'm trying to use awk to pretty up a log file, and calculate a percentage.

The log file looks like this:

[root@mys-nycnm-srv01 rtaStats]# tail strtovrUsage 
20090531-18:15:45  RSreq - 24, RSsuc - 24, RSrun - 78, RSerr - 0,
20090531-18:25:45  RSreq - 17, RSsuc - 17, RSrun - 72, RSerr - 0,
20090531-18:35:45  RSreq - 19, RSsuc - 19, RSrun - 72, RSerr - 0,
20090531-18:45:45  RSreq - 16, RSsuc - 16, RSrun - 60, RSerr - 0,
20090531-18:55:45  RSreq - 14, RSsuc - 14, RSrun - 62, RSerr - 0,
20090531-19:05:46  RSreq - 27, RSsuc - 26, RSrun - 61, RSerr - 1,
20090531-19:15:46  RSreq - 37, RSsuc - 37, RSrun - 74, RSerr - 0,
20090531-19:25:46  RSreq - 48, RSsuc - 46, RSrun - 84, RSerr - 2,
20090531-19:35:46  RSreq - 54, RSsuc - 54, RSrun - 106, RSerr - 0,
20090531-19:36:52  RSreq - 0, RSsuc - 0, RSrun - 107, RSerr - 0,

If I fun something simple like:

[root@mys-nycnm-srv01 ~]# cat `which SOperf`
 tail -f /usr/local/mystro/scripts/twctools/rtaStats/strtovrUsage|awk '{print "Requests = " $4 "Success = " $7  "Running Streams = " $10 "Errors = " $13 "Percentage = " ($7 / $4 )*100 }'

[root@mys-nycnm-srv01 ~]# SOperf 
Requests = 47,Success = 47,Running Streams = 122,Errors = 0,Percentage = 100
Requests = 62,Success = 60,Running Streams = 122,Errors = 2,Percentage = 96.7742
Requests = 56,Success = 56,Running Streams = 132,Errors = 0,Percentage = 100
Requests = 50,Success = 50,Running Streams = 124,Errors = 0,Percentage = 100
Requests = 65,Success = 63,Running Streams = 138,Errors = 0,Percentage = 96.9231

It works, but it's a little simpler than what I'd like. I tried doing the following:

#!/bin/bash

printf "Time\t\t\tRequests\tSuccess\tRunning\tError\tPercentage\n"

tail   /usr/local/mystro/scripts/twctools/rtaStats/strtovrUsage |\
awk         \
'***** = $1} \
 {req = $4} \
 {suc = $7} \
 {err = $13}\
 {run = $10}\
  END       \
{print tim "\t" req "\t\t" suc "\t" run "\t" err "\t" (suc / req)*100 }'

Time                    Requests        Success Running Error   Percentage
20090531-21:35:47       65,             64,     146,    1,      98.4615

But I only get the one line of results, and it won't work with a tail -f.
If it matters, here is the information for the system I'm working on.

Linux mys-nycnm-srv01 2.6.9-34.ELsmp #1 SMP Fri Feb 24 16:54:53 EST 2006 i686 i686 i386 GNU/Linux
/usr/bin/awk: symbolic link to `../../bin/gawk'

Also, is there an effective way to work in "divide by 0" error protection into what I'm trying to do (bearing in mind my obviously low skill level?)

1 Like
2

check for 0, or use >0

3

Thanks for the advice, shortly after posting I came up with some ideas using some basic if/else for the divide by 0 (though I've yet to try putting it into practice). I'm not sure what you mean by use >0 though?

Either way, my main concern is why the second method of running the script doesn't work, even though the first does. It seems piping tail to awk works, but only sometimes.

4

check that denominator and numerator (or rather divisor, and divider?? forgot the english) are both more than 0, otherwise, how to divide, right? so do a simple if/else...

5

Yeah, for once I actually had the basic knowledge to make it work, for some reason the logic just escaped me at the time. I came up with this:

[root@mys-nycsm-srv01 rtaStats]# cat SOperf 
#!/bin/bash

tail -f /usr/local/mystro/scripts/twctools/rtaStats/strtovrUsage |\
        awk \
        '{if ($4 == "0," || $7 == "0,") \
                print \
                        $1,
                        "Requests = " $4 \
                        "Success = " $7  \
                        "Running Streams = " $10 \
                        "Errors = " $13 \
                        "Percentage... if I tried to get you a percentage the world would explode " \
         ;else \
                print \
                        $1,
                        "Requests = " $4  \
                        "Success = " $7   \
                        "Running Streams = " $10  \
                        "Errors = " $13  \
                        "Percentage = " ($7 / $4 )*100 }'

[root@mys-nycsm-srv01 rtaStats]# SOperf 
20090601-04:03:46 Requests = 5,Success = 5,Running Streams = 17,Errors = 0,Percentage = 100
20090601-04:13:46 Requests = 3,Success = 3,Running Streams = 12,Errors = 0,Percentage = 100
20090601-04:23:47 Requests = 4,Success = 3,Running Streams = 11,Errors = 1,Percentage = 75
20090601-04:33:47 Requests = 1,Success = 0,Running Streams = 7,Errors = 1,Percentage... if I tried to get you a percentage the world would explode 
20090601-04:43:47 Requests = 1,Success = 1,Running Streams = 7,Errors = 0,Percentage = 100
20090601-04:53:47 Requests = 2,Success = 2,Running Streams = 4,Errors = 0,Percentage = 100
20090601-05:03:47 Requests = 4,Success = 4,Running Streams = 4,Errors = 0,Percentage = 100
20090601-05:13:47 Requests = 5,Success = 5,Running Streams = 5,Errors = 0,Percentage = 100
20090601-05:23:47 Requests = 4,Success = 4,Running Streams = 4,Errors = 0,Percentage = 100
20090601-05:33:47 Requests = 4,Success = 4,Running Streams = 5,Errors = 0,Percentage = 100

I'm still confused though. While playing around with the other version of the script I made, I made the minor change:

#!/bin/bash

printf "Time\t\t\tRequests\tSuccess\tRunning\tError\tPercentage\n"

tail -f  /usr/local/mystro/scripts/twctools/rtaStats/strtovrUsage |\
awk         \
'***** = $1} \
 {req = $4} \
 {suc = $7} \
 {err = $13}\
 {run = $10}\
{print tim "\t" req "\t\t" suc "\t" run "\t" err "\t" (suc / req)*100 }'
[root@mys-nycnm-srv01 ~]# ./xperf 
Time                    Requests        Success Running Error   Percentage
20090601-04:05:51       5,              5,      21,     0,      100
20090601-04:15:51       5,              5,      19,     0,      100
20090601-04:25:51       8,              8,      22,     0,      100
20090601-04:35:51       7,              7,      21,     0,      100
20090601-04:45:51       3,              3,      17,     0,      100
20090601-04:55:51       10,             10,     16,     0,      100
20090601-05:05:51       5,              4,      16,     1,      80
20090601-05:15:52       8,              0,      16,     7,      0
20090601-05:25:52       14,             0,      12,     14,     0
20090601-05:35:52       15,             1,      12,     14,     6.66667

And now it runs perfectly. It even takes care of the divide by zero issue with no additional logic. Admittedly the "END" part was only there because I was cutting and pasting from other things I've pieced together, and not a real understanding of what it means, but could anyone give me a brief explanation of what END was doing?