Grep a log file for the last 5 minutes of contents every 5 minutes

Hi all,
System Ubuntu 16.04.3 LTS
i have the following log

INFO     2019-02-07 15:13:31,099 module.py:700] default: "POST /join/8550614e-3e94-4fa5-9ab2-135eefa69c1b HTTP/1.0" 500 2042
INFO     2019-02-07 15:13:31,569 module.py:700] default: "POST /join/6cb9c452-dcb1-45f3-bcca-e33f5d450105 HTTP/1.0" 500 2042
INFO     2019-02-07 15:14:25,379 module.py:700] default: "POST /join/f30c767d-fcd2-44c2-83a9-d927fv10f7a8 HTTP/1.0" 500 2042
INFO     2019-02-07 15:14:46,531 module.py:700] default: "POST /join/57e7930c-99ea-419a-b807-fad6327cd6e9 HTTP/1.0" 500 2042
INFO     2019-02-07 15:15:06,966 module.py:700] default: "POST /join/57e7930c-99ea-419a-b807-fad6447cd6e9 HTTP/1.0" 500 2042
INFO     2019-02-07 15:15:06,966 module.py:700] default: "POST /join/57e793rfc-98ea-418a-b817-fad6347cd6e9 HTTP/1.0" 500 2042

i need to create a monitor for this log and grep only last 5 minutes of this log HTTP/1.0" 500 2042, and make an action if the match count is higher then 5 in last 5 minutes.

I tyied the following script but for some reason i can't get it working

#!/bin/bash
#
check=$(awk -v d1="$(date --date="-5 min" "+%Y-%m-%d %H:%M:%S,%3N")" -v d2="$(date "+%Y-%m-%d %H:%M:%S,%3N")" '$0 > d1 && $0 < d2 || $0 ~ d2' /var/log/messages | grep -i "HTTP/1.0" 500 2042")
count=$(awk -v d1="$(date --date="-5 min" "+%Y-%m-%d %H:%M:%S,%3N")" -v d2="$(date "+%Y-%m-%d %H:%M:%S,%3N")" '$0 > d1 && $0 < d2 || $0 ~ d2' /var/log/messages | grep -ci "HTTP/1.0" 500 2042") 
if [ $count -gt 5 ] 
then
   echo $CHECK | /bin/mail -s "$count occurrences of the error message has been found in the last 5 minutes" myemail@mydomain.com 
else
   echo "Exit, everything is ok" > /dev/null
fi

The error i gues is with grep because there are the " in the middle.
Could somebody help me on this what i'm i missing here?
Thank you in advance.

I do not know how it should work without a plus sign ?

date --date="-5 min" "%Y-%m-%d %H:%M:%S,%3N"

Yes, sorry i wrote this without the +, this is a typo i made here, but the original copy has is, and it doesn't work either.
Thank you.

Here to begin with, and I will try to understand

awk -F, -v d="$(date --date="-5 minutes" +"%Y-%m-%d %H:%M:%S,%3N")" '( d > substr($1, 10)) { print }' file

--- Post updated at 12:38 ---

Just need to escape quotes

grep -i "HTTP/1.0\" 500 2042"

--- Post updated at 12:45 ---

And rearrange options. Like this
grep -i

--- Post updated at 13:03 ---

the awk itself well knows how to count

awk -F, -v d="$(date --date="-50 minutes" +"%Y-%m-%d %H:%M:%S,%3N")"
'$0 ~ "HTTP/1.0\" 500 2042$" {if ( d > substr($1, 10)) error++} END {print error}' file

or use different quotes, to avoid ugly escaping...

grep -i 'HTTP/1.0" 500 2042'

I would omit the fixed request size in the grep too, maybe that changes at any time(software upgrade?)

grep -i 'HTTP/1.0" 500 [0-9]+'

Hello nezabudka,
this code is printing all the matchs on the file, and not only the last five minutes of the file,
any idea?

awk -F, -v d="$(date --date="-5 minutes" +"%Y-%m-%d %H:%M:%S,%3N")"
'$0 ~ "HTTP/1.0\" 500 2042$" {if ( d > substr($1, 10)) error++} END {print error}' file

I was mistaken

( d > substr($1, 10))

true

( d < substr($1, 10))

It doesn't work neither,
any other ideas?

It works for me with your log file. Try for example to increase the time interval from 5 minutes to 500

--- Post updated at 15:05 ---

comment out all lines leave only this in the script

awk -F, -v d="$(date --date="-5000 minutes" +"%Y-%m-%d %H:%M:%S,%3N")" \
'$0 ~ "HTTP/1.0\" 500 2042$" {if ( d < substr($1, 10)) error++} END {print error}' file

If not working, try to change the filter or index substr 10

awk -F, -v d="$(date --date="-5000 minutes" +"%Y-%m-%d %H:%M:%S,%3N")" \
'$0 ~ "500 2042$" {if ( d < substr($1, 10)) print}' file

#!/bin/sh
check=$(awk -F, -v d="$(date --date="-5 minutes" +"%Y-%m-%d %H:%M:%S,%3N")" \
'$0 ~ "HTTP/1.0\" 500 2042$" {if ( d < substr($1, 10)) error++} END {print error}' /tmp/test.log)

if [ $check -gt 5 ]
then
   echo "Bad News, Need to send Alert"
else
   echo "Exit, everything is ok"
fi

So i have the following error now,

sh -x test.sh
+ date --date=-5 minutes +%Y-%m-%d %H:%M:%S,%3N
+ awk -F, -v d=2019-02-08 15:12:29,459 $0 ~ "HTTP/1.0\" 500 2042$" {if ( d < substr($1, 10)) error++} END {print error} /tmp/test.log
+ check=
+ [ -gt 5 ]
test.sh: 5: [: -gt: unexpected operator
+ echo Exit, everything is ok
Exit, everything is ok

Any ideas?
Thank you.

I again made a mistake, did not take nanoseconds into account. Remove them from the template

date --date="-5 minutes" +"%Y-%m-%d %H:%M:%S,%3N"

Together with a comma. They are not needed

--- Post updated at 16:08 ---

awk -F, -v d="$(date --date="-500minutes" +"%Y-%m-%d %H:%M:%S")" \
        '$0 ~ "HTTP/1.0\" 500 2042$" {if ( d < substr($1, 10)) print}' file

Still having the same issue

test.sh: 5: [: -gt: unexpected operator
+ echo Exit, everything is ok

How do i fix this?
Thanks

The check= indicates something went wrong with your awk command as it delivered nothing on its stdout to be assigned to the variable. So

• why the "double quotes in double quotes" for the date command?
• why the extra precision in the date format?
• why the comma field separator?
• why the complicated matching?How about

check=$(awk -v d="$(date --date='-5 minutes' +'%Y-%m-%d %H:%M:%S')" '/HTTP\/1.0" 500 2042$/ {if ( d < $2 " " $3) error++} END {print error}' /tmp/test.log)

And, while we're at it, why not have awk process the entrie stuff for us? Like

awk -v d="$(date --date='-5 minutes' +'%Y-%m-%d %H:%M:%S')" '
/HTTP\/1.0" 500 2042$/  {if ( d < $2 " " $3) error++
                        }
END                     {if (error >= 5) print "/bin/mail -s \"" error " occurrences ...\" myemail@mydomain.com"
                           else          print "echo \"Exit, everything is ok\""
                        }
'  /tmp/test.log

You'll see the mail or echo happy output depending on your relevant log file error count. If happy with what you see, pipe the output through sh so the resp. command will be executed.

Hello Rudic,
yes your suggestion work also,
to fix my previous issue i hade to add if there were no match at all,
and that worked just fine

if [ -z  $check ]
        then
echo "Exit, everything is ok"
        exit
fi
if [ $check -gt 5 ]
        then
   echo "Bad News, Need to send Alert"
        else
   echo "Exit, everything is ok"
fi

Thank to all for the support.

To make sure a result is printed and check is assigned a value, even if no error occurred, make the END section read

END {print error + 0}

and, eventually, append an exit 1 in the error case so you can evaluate the exit code and react accordingly.