Searching exception keyword in all logs in last 5 minutes

UNIX for Beginners Q & A
1

Hello Folks ,

I am a new bie to the world of unix , what i am planning to do is the I have the location in server to which i am access through the putty and the location is

/mt/ttlog/avccomn/logs/201901/19

and at this location the files are listed as show

startjmsnode1.sh_03.out
startmainuiclientnode1.sh_02.out
startmaincommon_moon2.sh_01.out

Now please advise the unix command which can extract the last 5 minutes logs that i want to search the keyword Exception if it is there in last 5 minutes in logs.

2

For this we would have to know the contents of said files. Usually logs are files that are written to constantly and what would be written within these last fieve minutes we (and all others, for that matter) don't know - without looking into it.

So post a sample of the files contents and maybe we can find some pattern by which to discern "within-the-last-five-minutes"- and "older"-types of messages.

At any rate, you can use the grep utility to search for a certain pattern (like the word "exception") in a file. The following will search for either "exception" or "Exception" in a file and display all lines containing it to the screen:

grep "[Ee]xception" /path/to/file/to/examine

I hope this helps.

bakunin

3

Hello Folks ,

I am a new bie to the world of unix , what i am planning to do is the I have the location in server to which i am access through the putty and the location is

/mt/ttlog/avccomn/logs/201901/19

and at this location contain the various log files some of them are listed as show

startjmsnode1.sh_03.out
startmainuiclientnode1.sh_02.out
startmaincommon_moon2.sh_01.out

Now please advise the unix command which can extract the last 5 minutes logs from all the different logs listed in this particular location , as i want to search the keyword Exception if it is there in last 5 minutes in all logs.

Below is the way that logs have been written as I have open them in VI editor , also please cross check the timestamp fashion along with each line

OMM 2018-11-29T02:07:40,710 DEBUG AM-EventNotificationCron  decrypt Entry
OMM 2018-11-29T02:07:40,710 DEBUG AM-EventNotificationCron  getEightByteIV Entry
OMM 2018-11-29T02:07:40,710 DEBUG AM-EventNotificationCron  getEightByteIV srciv == null (salt)
OMM 2018-11-29T02:07:40,712 DEBUG AM-EmailLogCron  (EmailLogCron.sendEmailActivity) SQL Error in abc.....
OMM 2018-11-29T02:07:40,713 DEBUG AM-abcLogCron  java.sql.SQLException: Numeric Overflow

--- Post updated at 05:19 AM ---

folks please advise for the above problem.

4

Try this :

#!/bin/sh
cd /mt/ttlog/avccomn/logs/201901/19
e=
for (( i = 5; i >= 0; i-- )) ; do
    e='-e /'`date +\%R -d "-$i min"`'/p '$e
done

$(sed -n $e startjmsnode1.sh_03.out > newfile)
1 Like
5

@punpun26262626: Welcome to the forum.
It is usually well received in here if you put some more effort into formulating the spec than "Now please advise". What OS / shell / tools versions do you use? What thoughts / logics lead to the desired result? Any attempts from your side on the solution?

@sadique.manzar: nice idea, with three drawbacks:

  • the patterns will match more than "last 5 minutes", e.g. the day(s) before, or matching "min:sec" values (in above: /07:40/), or any other similar data.
  • the key word "Exception" is requested.
  • no "command substitution" for the final sed command.

Combining the two proposals this far we come up with

$ e="\("
$ DL=
$ for (( i = 5; i >= 0; i-- )) ; do  e=$(date +"$e$DL%dT%R:" -d "-$i min"); DL="\|"; done
$ e=$e"\).*[Ee]xception"
$ grep $e file
OMM 2018-11-29T02:07:40,713 DEBUG AM-abcLogCron  java.sql.SQLException: Numeric Overflow
1 Like
7

Perhaps the question is if the log entries are sorted by timestamp already. If they are we could simply use:

sed -n '/[Ee]xception/p;/<timestamp more than 5 min away>/q' /path/to/log

and calculate a "threshhold timestamp" up front.

I hope this helps.

bakunin

8

Even IF those were sorted, the approach with the single threshold timestamp would require that value to occur verbatim in the logs, so the latter need at least one entry per minute. And, wouldn't the log file need to be tac ced to retrieve the last five minutes, and then quit?

9

Note that punpun26262626 says that he or she is running commands on a UNIX system through putty. Since, date -d is a GNU date extension, there is a good chance that none of the above suggestions will work.

Hello punpun26262626,
When starting a new thread on unix.com, please always tell us what operating system and shell you're using. Saying you're using UNIX is roughly equivalent to saying that you're driving a vehicle when someone asks you what model of car, truck, or motorcycle you drive. Asking questions about handling strings containing dates is one of the topics where knowing exactly what versions of operating system you're using and which shell you're using are crucial to getting help that will work in your environment.

Please help us help you.