Help with scritp to count an specific word into a log

Shell Programming and Scripting
1

Hello my friends, I need to count how many words are into a log file, I'm using:

cat logfile | grep 'word' | wc -l

Cuz the 'word' appears once per line.

But my logfile grow faster and at the end ofthe day is really big, so how i can count the 'word' only from (by example) line 4000 of the logfile til the end, then get the last number of line (x) and in the next time count from 'x' line til the new end (x'), that has sence?

Thanx for your help.

Lestat

2

what you can do is capture the line count to a file.
then whenever you run the script, get the value into a shell variable. with this variable, you can do your processing..

3

Firstly, you don't need to use cat and wc -l while searching. You can simply use

 grep -c 'word'  logfile

For searching from a particular line number onwards, you can save the line number in a variable and pass it to gawk or nawk:

 awk -v num=${saved_line_no} -v word=${word} 'NR>num && $0~word{ count++} END{ print count}' logfile
4

You can do something like this :

#!/usr/bin/ksh

#
# Define logile and searched word
#

log_file=logfile
searched_word=word
prv_line_count_file=${log_file}.prv_count

#
# Get previous line count
#

if [ -f ${prv_line_count_file} ]
then
   prv_line_count=$(<${prv_line_count_file})
else
   prv_line_count=0
fi

#
# Get actual line count and memorize it
#

line_count=$(awk 'END {print NR}' ${log_file})
echo ${line_count} > ${prv_line_count_file}

#
# If logfile is smaller, start searching at the first line
#

if [ ${prv_line_count} -gt ${line_count} ]
then
   prv_line_count=0
fi

#
# Search for word
#

awk -v num=${prv_line_count} \
    -v word="${searched_word}" \
    'NR>num && $0~word { count +=1 } END { print count+0 }' \
    ${log_file}

jean-Pierre.

5

You could do something much simpler:

tail -1000 filename | grep 'word' | wc -l

This will count the the number of lines that word appears in the last 1000 lines of the file (or less if there are not 1000 lines in the file).

So, you could keep a track of the line that you last searched up to, and get the number of lines (wc -l filename) and stick the difference into the -1000 bit.

6

you have revived an old thread. pls check before posting.