Script to check response time from nginx logs

Shell Programming and Scripting
1

Hi,

My goal is to monitor the response time from the access logs of nginx server. I am using gawk to print the needed fields - 'response time' and 'name of the service' from nginx logs.

Command: gawk '($6 ~ /cloudservice/) {print $10, $6}' access.log

Output:
0.645 /nc/cloudservice
0.041 /nc/cloudservice
1.346 /nc/cloudservice
0.020 /nc/cloudservice

So, I have the response time for this particular service. Now, I want to compare the response time with a number (say 1.000 in this case) and get the result only with response time > 1.000 and send an email with the output.

Is this possible in a single awk/gawk statement?

Thanks,
Naitik

2 
gawk '$6 ~ /cloudservice/ && $10+0 > 1.000 { print $10, $6 }' access.log | mailx -s "Subject" user@domain.com
1 Like
3

Wow Yoda!
Thanks for the quick response. It works like a charm. :slight_smile:

---------- Post updated at 06:26 PM ---------- Previous update was at 04:37 PM ----------

I have a question on the same - suggestions are welcome.

The access.log file is not a static file - the server log keeps on populating the file (Also, there is no log rotation on the file).

If I use this command and run it every minute, I get redundant records.

Command:

gawk '$6 ~ /cloudservice/ && $10+0 > 1.000 { print $10, $6 }' access.log | mailx -s "Subject" user@domain.com

Output:
1.346 /nc/cloudservice

I get this output everytime I run the command. How do I just get the new values in output? The access log file has a date and time stamp for each line.

I am not quite sure how to approach this. Any suggestions?

Thanks,
nshah11

4

You could have your script "remember" the last line examined in a file eg /srv/nginx_monitor/last_line

LAST_FILE=/srv/nginx_monitor/last_line

END_LINE=$(wc -l < access.log)
START_LINE=0

if [ -f "$LAST_FILE" ]
then
    read START_LINE < "$LAST_FILE"
    # Check if file has shrunk - could have been truncated
    [ $END_LINE -lt $START_LINE ] && START_LINE=0
fi

gawk -v S=$START_LINE -v E=$END_LINE '
    # Ignore anything appended after we did count
    NR>E{exit}
    NR>=S && $6 ~ /cloudservice/ {print $10, $6}' access.log

echo "$END_LINE" > "$LAST_FILE"
1 Like
5

START_LINE is the last END_LINE, so it is rather

    (NR>S && $6 ~ /cloudservice/ && $10+0 > 1.0) {print $10, $6}' access.log

together with the first requirement.

2 Likes
6

Thanks MadeInGermany another out-by-one, I swear it feels like 49% of my errors are of this type.

7

Thanks Chubler_XL and MadeInGermany for your inputs.

Seemed a little complicated to understand since I am a newbie. Currently, I am using two files and finding the difference of the contents (to find the newly appended records). Have not tested it yet - shall keep you posted once I try it out.

Thanks!