I am creating a shell script which will check the processing of the main script if the error count is more than 2% of the record count in a feed. Part of the code is below:
err_tol=`echo $feed_cnt \* 0.02 |bc`
while read line
do
err_cnt=$(grep -i "$line" $err_log | wc -l)
if [ $err_cnt > $err_tol ]
then
echo "Errors exceed the tolerance level"
exit 1
fi
done <"$err_list"
feed_cnt - is the word count of the incoming feed err_list - is a file with all errors that can be raised by the process
I am using bc in err_tol calculation as I need to multiply decimal numbers. This statement is creating blank files with the divided number value.
I have tried a few other methods and they seem to be creating blank files as well.
Is err_list the name of a file, or is it the name of a variable that contains the name of a file?
Are you trying to see if there are more than 2% of a specific type of error as specified by one line in the file named by $err_list , or all types of errors specified by all of the lines in the file named by $err_list ?
How is feed_cnt set?
The command: text [ $err_cnt > $err_tol ]
creates a file named by the expansion $err_tol , and stores the output of the command: text [ $err_cnt ]
in that file. If you were trying to do a numeric comparison, that would be: text [ "$err_cnt" -gt "$err_tol" ]
but (unless you're using a 1993 or later version of ksh) that only works for integer values (and, if you're using a 1993 or later ksh, you don't need to use bc to do any of there calculations used here).
What errors are you getting from your script?
-- I am not getting any error from the script as such and is working as expected.
Is err_list the name of a file, or is it the name of a variable that contains the name of a file?
-- err_list is the name of a file which contains different error types a file can have.
Are you trying to see if there are more than 2% of a specific type of error as specified by one line in the file named by $err_list , or all types of errors specified by all of the lines in the file named by $err_list ?
-- I want to if all types of errors specified by all of the lines in the file name $err_list is more than 2%
How is feed_cnt set? -- This value is wordcount on the file (wc -l)
Changing the condition to -gt eliminated creating the blank files.
Any thoughts on how I can check the sum of all types of errors is more than 2% of the feed count?
awk -v fc=$feed_cnt 'NR==FNR { f[$0]++; next } $0 in f { total += f[$0] }; END { if (total > (0.02 * fc)) print "Errors exceed the tolerance level" }' $err_log $err_list
You didn't say which version of UNIX you're using and you didn't say which version of ksh you're using. If you just want to use ksh and wc (instead of awk ), you can also try the following:
#!/bin/ksh
#set -xv
err_log="err_log"
err_list="err_list"
feed_cnt=$(wc -l < "$err_log")
err_tol=$(((feed_cnt + 25) / 50))
# grep -iFf "$err_list" "$err_log"
err_cnt=$(grep -iFcf "$err_list" "$err_log")
if [ "$err_cnt" -gt "$err_tol" ]
then
echo "Errors exceed the tolerance level"
exit 1
fi
Note that Aia's awk script looks for exact (case sensitive) full line matches; the code above looks for case insensitive matches where the complete contents of any line from err_list appears anywhere on a line in err_log.
If you want the behavior of the script above, but want to use awk instead, the changes to Aia's awk script to get that behavior are minor.
Since you didn't give us any sample data, I tried to match the options you gave to grep . But, I assumed you were looking for fixed strings rather than having a list of basic regular expressions in err_list.
If we guessed wrong, give us details of your real requirements.
If you're using a 1993 or later version of ksh , we can use floating point arithmetic to compute err_tol instead of playing games with rounding using integer arithmetic.
If you have an old UNIX system with a grep that doesn't accept the -F option, change grep -iFcf to fgrep -icf .