Hi Guys,
I would like to create a script which tails the content of a log file in real time, looks for a specific string , like "ERROR" and captures in a text file the previous 10.000 lines that were existing before this string.
Any help is appreciated.
This is not possible by tail command to predict when ERROR is coming and capture the 10 lines before that, instead of this, script can read the file in a loop every time where it left and parse the file up to next error and get the previous 10 line. let me get the some raw script for you.
This could be the foundation of such a script
while true ; do
tail $logfile | grep ERROR || tail -10000 $logfile > prev &&echo "Dumped log records to prev"&& break ; done
Let me make it more clear for you.
I am using some application that puts all the logging under /var/adm/messages. Once i capture the string "Error" I need to find out what actually caused it, 1000 lines before and after, to have the full picture.Then put all these lines in a text file.
It's going to be something like this :
$logfile= /var/adm/messages
while true ; do tail $logfile | grep ERROR || tail -1000 $logfile > prev file.txt &&echo "Dumped log records to prev"&& break ; done
---------- Post updated at 04:24 PM ---------- Previous update was at 03:03 PM ----------
Hi,
Is my above expression correct ?
Thanks!
The script posted contains too many syntax errors to make the intention clear.
What Operating System and version do you have and what Shell are you using?
try this.
#!/bin/bash
previousSearch=0
while true;
do
Lines=$(cat /var/adm/messages | grep -n ERROR | cut -d ":" -f1) #to get the all lines numbers which has word ERROR
for i in $Lines
do
if [ $i -gt previousSearch ]; then
line=$(expr $i + 1000) #get the maximun number of line which is needed
head -$line | tail -2000 >> newfile.txt # get exact 2000 line, 1000 before ERROR and 1000 after ERROR
previousSearch=$i # setting counter to last search result.
fi
done
done
I was not able to run this but if there is any syntax error please fix thos, but logic seems to be good.
you run this script as background process and tail the newfile.txt to get the result.
---------- Post updated at 10:44 AM ---------- Previous update was at 10:34 AM ----------
in fact you can use this line also instead of original
Lines=$\(grep -n ERROR /var/adm/messages | cut -d ":" -f1\) \#to get the all lines numbers which has word ERROR
This would be must efficient then original "cat".
I'm still getting an error when running the script :
[: previousSearch: bad number
The error message is because there is a $ missing off $previousSearch:
if [ $i -gt $previousSearch ]; then
Livisbr, As i said i haven't tested and expected you to fix the run time issue if any... I have given you almost ready script.
here is the script with some fixes.
#!/bin/bash
previousSearch=0
while true;
do
Lines=$(cat /var/adm/messages | grep -n ERROR | cut -d ":" -f1) #to get the all lines numbers which has word ERROR
for i in $Lines
do
if [ $i -gt $previousSearch ]; then
line=$(expr $i + 1000) #get the maximun number of line which is needed
head -$line /var/adm/messages | tail -2000 >> newfile.txt # get exact 2000 line, 1000 before ERROR and 1000 after ERROR
previousSearch=$i # setting counter to last search result.
echo "---------------------------Next Search---------------------------" >> newfile.txt
fi
done
sleep 30
done
Beware of using large values in a unix "tail" command. See "man tail" for limitations of the buffer size in "tail" on your system.
wc -l mylog
71520 mylog
tail -2000 mylog|wc -l
394
Right Methyl...
livisbr, do you really need 1000 lines above and below the error? you can make it 100 or 200 that should be enough for you to get the cause of error.
I hope this is done now.
Why dont you try
print n number of lines after the line containing ERROR
print n number of lines before the line containing ERROR