Extract a value from log

Hi,

I have a log file:

130.37.129.78 - - [25/Jun/2011:12:35:07 +0000] "GET /ganglia/graph.php?m=load_one&z=small&c=cluster1&h=xen02.das3.cs.vu.nl&l=e2ecff&v=0.00&x=0.22&n=0&r=hour&su=1&st=1309005296 HTTP/1.0" 200 6818 "http://130.37.197.12/ganglia/index.php" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:5.0) Gecko/20100101 Firefox/5.0" request:0.265

I would like to extract the value next to the string request: from the file. Could someone help me with this. :slight_smile:

awk -F: '{print $NF}' logfile
1 Like

Thank you, but the thing is I am extracting the value line by line from the log.

/***************************************/
cat $LOG | while read LINE
do
# Obtain the value time from each request
REQUEST=?

/******************************************/

Could you tell me how to modify the above command for this purpose.

please post an example of your input as well as an example of the output you need

---------- Post updated at 02:39 PM ---------- Previous update was at 02:36 PM ----------

$ cat logfile
130.37.129.78 - - [25/Jun/2011:12:35:07 +0000] "GET /ganglia/graph.php?m=load_one&z=small&c=cluster1&h=xen02.das3.cs.vu.nl&l=e2ecff&v=0.00&x=0.22&n=0&r=hour&su=1 &st=1309005296 HTTP/1.0" 200 6818 "http://130.37.197.12/ganglia/index.php" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:5.0) Gecko/20100101 Firefox/5.0" request:0.265
130.37.129.78 - - [25/Jun/2011:12:35:07 +0000] "GET /ganglia/graph.php?m=load_one&z=small&c=cluster1&h=xen02.das3.cs.vu.nl&l=e2ecff&v=0.00&x=0.22&n=0&r=hour&su=1 &st=1309005296 HTTP/1.0" 200 6818 "http://130.37.197.12/ganglia/index.php" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:5.0) Gecko/20100101 Firefox/5.0" request:0.857
130.37.129.78 - - [25/Jun/2011:12:35:07 +0000] "GET /ganglia/graph.php?m=load_one&z=small&c=cluster1&h=xen02.das3.cs.vu.nl&l=e2ecff&v=0.00&x=0.22&n=0&r=hour&su=1 &st=1309005296 HTTP/1.0" 200 6818 "http://130.37.197.12/ganglia/index.php" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:5.0) Gecko/20100101 Firefox/5.0" request:0.376
$ awk -F: '{print $NF}' logfile
0.265
0.857
0.376
$

Hi,

The input is a nginx log file with many requests of this syntax:

130.37.129.78 - - [25/Jun/2011:12:55:10 +0000] "GET /ganglia/graph.php?m=load_one&z=small&c=cluster1&h=xen02.das3.cs.vu.nl&l=e2ecff&v=0.00&x=0.32&n=0&r=hour&su=1&st=1309006507 HTTP/1.0" 200 7085 "http://130.37.197.12/ganglia/index.php" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:5.0) Gecko/20100101 Firefox/5.0" request:0.252

I am parsing this log file line by line and extracting the request: value from each line and computing their average. Could you help me as to how to extract the value after the "request:" string from each line.

With the same example file as above :

$ awk -F: '{++x;s+=$NF;print $NF}END{print "average=" s/x}' logfile
0.265
0.857
0.376
average=0.499333

---------- Post updated at 02:47 PM ---------- Previous update was at 02:43 PM ----------

I need to know how your logfile looks like .

Does it only contain lines that ends with a string "request:<value>" ?

If not then please post a representative example of how does your logfile look like (with all the lines, not only those that have the "resquest:<value>", so we can then be more accurate in the line selection before extracting the value and calculating the average

1 Like

Wow, that is perfect. Just what I needed in one line :slight_smile:

One more thing, is there a way to store the value of average in another variable, since I am having a shell script which runs the above script inside it, so I would like to store the average value into a variable.

If you are only interested in the average value then :

average=$(awk -F: '{++x;s+=$NF}END{print s/x}' logfile)