How to count respon time max min avg for nginx logs?

UNIX for Beginners Q & A
1

Hi All,

need your help, i want count respon time max and average my nginx logs, based on hourly or minutes per api...

my nginx.log sample :

10.1.1.1 - - [25/Aug/2019:05:26:30 +0700] "POST /v2/api/find/outlet/ HTTP/1.1" 200 2667 "-" "okhttp/3.12.0" "118.215.153.47" 0.178 0.178 .
10.1.1.1 - - [25/Aug/2019:05:26:30 +0700] "POST /v2/api/find/outlet/ HTTP/1.1" 200 2847 "-" "okhttp/3.12.0" "189.246.151.188" 0.177 0.178 .
10.1.1.1 - - [25/Aug/2019:05:27:52 +0700] "GET /v2/api/menu/category HTTP/1.1" 401 40 "-" "okhttp/3.12.0" "139.194.84.246" 0.007 0.007 .
10.1.1.1 - - [25/Aug/2019:05:27:52 +0700] "GET /v2/api/user/point HTTP/1.1" 200 152 "-" "okhttp/3.12.0" "202.80.217.172" 0.028 0.028 .
10.1.1.1 - - [25/Aug/2019:05:27:52 +0700] "GET /v2/api/user/destination HTTP/1.1" 200 169 "-" "okhttp/3.12.0" "36.91.42.35" 0.019 0.019 .
10.1.1.1 - - [25/Aug/2019:05:28:52 +0700] "POST /v2/api/transaction/inquiry HTTP/1.1" 200 503 "-" "okhttp/3.12.0" "36.89.234.129" 0.374 0.374 .
10.1.1.1 - - [25/Aug/2019:05:28:52 +0700] "POST /v2/api/transaction/confirm HTTP/1.1" 200 874 "-" "okhttp/3.12.0" "36.89.234.129" 0.394 0.394 .
10.1.1.1 - - [25/Aug/2019:05:28:52 +0700] "GET /v2/api/user/point HTTP/1.1" 200 152 "-" "okhttp/3.12.0" "114.5.147.117" 0.024 0.024 .
10.1.1.1 - - [25/Aug/2019:05:28:52 +0700] "GET /v2/api/menu/category HTTP/1.1" 403 40 "-" "okhttp/3.12.0" "139.194.84.246" 0.003 0.003 .

my expectation sample below :

date                       |  api                               | max| avg

25/Aug/2019:05:26 /v2/api/find/outlet             2847 2757

25/Aug/2019:05:27 /v2/api/menu/category HTTP/1.1               1847 1757
25/Aug/2019:05:28 /v2/api/menu/category HTTP/1.1               1147 1257

i already trying with this awk but only qot average :

awk '/25\/Aug\/2019:18/ {c++} END{print c}' access.log

please help ...

Thanks

2
  • start with:
awk '
$4 ~ /[0-9]:[0-9]/ {
date=$4; sub(/^[^0-9]*/, "", date); sub(/:[0-9][0-9]$/, "", date);
id=date OFS $7; api[id]; apic[id]++; apit[id]+=$10; if ($10>apim[id]) apim[id]=$10;
date=$4; sub(/^[^0-9]*/, "", date); sub(/:[0-9][0-9]:[0-9][0-9]$/, " ", date);
id=date OFS $7; api[id]; apic[id]++; apit[id]+=$10; if ($10>apim[id]) apim[id]=$10;
}
END {
print "date/time" OFS "api" OFS "max" OFS "avg"
for (i in api) print i OFS apim OFS apit / apic;
}
' OFS="\t" access.log | sort -r
3

Wouldn't you need to sum up values to calculate an average?

If you want hourly averages - as indicated - , try this slight modification of rdrtx1's proposal:

awk '
$4 ~ /[0-9]:[0-9]/      {date=$4
                         sub(/^[^0-9]*/, "", date)
                         sub (/:[0-9]*:[0-9]*$/, "", date)                  # get rid of mins and secs for the hourly stuff
                         id=date OFS $7
                         apic[id]++
                         apis[id] += $10                                    # sum up values for later avg calc.
                         if ($10>apim[id]) apim[id] = $10;
}
END                     {print "date" OFS "api" OFS "max" OFS "avg"
                         for (i in apic) print i OFS apim OFS apis / apic;
                        }
' OFS="\t" file
4

Hi rdrtx1

thanks for reply...your awk will generate per second right?

--- Post updated at 07:46 PM ---

Hi RudiC

Thanks Alot for your response...

this awk use for hour right? how to convert to minutes?

Thanks
Fajar

5

See updated of post # 3. mins, hrs max and avg included.