Variable scope in bash

Shell Programming and Scripting
1

Hello! Before you "bash" me with

  • Not another post of this kind
    Please read on and you will understand my problem...

I am using the below to extract a sum of the diskIO on a Solaris server.

#!/bin/sh
PATH=/usr/bin:/usr/sbin:/sbin; export PATH
TEMP1="/tmp/raw-sar-output.txt$$"
TEMP2="/tmp/sorted-sar-output.txt$$"
CONT=0
sar -d 30 4 > $TEMP1
AVG_START=`awk '/^Average/ {print NR}' $TEMP1`
TOT_LINES=`cat $TEMP1 |wc -l`
TAIL_LINES=`expr $TOT_LINES - $AVG_START`
tail -$TAIL_LINES $TEMP1 | awk '{ print $5,$2 }' | sort -k 1n,1 > $TEMP2
if [ -s "$TEMP2" ]
then
   while read file;
   do
      IO=`echo $file | awk '{ print $1 }'`
      TMP_VAL=`echo $file | awk '{ print $1 }'`
      CONT=`expr $CONT + $TMP_VAL`
   echo CONT = $CONT
   done < $TEMP2
   #Echo Result
   echo $CONT
else
   rm -f $TEMP1 $TEMP2 2>/dev/null
   exit 1
fi
rm -f $TEMP1 $TEMP2 2>/dev/null
exit 0

Even though I am not piping to while, the final

echo $CONT

returns zero.

The same code with Korn shell works perfectly but for some reason the execution is too slow and causes problems to the use I need it for. So I definately need to use Bash or something equally fast...

Please help!

1 Like
2

The execution is slow because you're running dozens of externals in backticks all the time. You should use externals to process big batches of data -- not individual lines. Externals are fast enough on bulk data, but overkill for anything less, like makng 10 phone calls to say 10 words. Most of your script could probably be done with one execution of awk, if we knew the details.

If you show the input you get from sar, and the output you want from the script, we can find a much more efficient way to do so, probably in nearly pure awk. It can do much more than '{ print $1 }'

I don't see anything obviously wrong with your script, though there's many places in it where things are unquoted and quoting may be important.

3

The script is not slow for its content.
The "sar -d 30 4" statement will take 2 minutes (sample interval of thirty seconds times the four iterations). On my test the script took 2 mins 3 secs.

The process to fish out the Average lines can be improved such that we only read $TEMP1 once.

#!/bin/sh
PATH=/usr/bin:/usr/sbin:/sbin; export PATH
TEMP1="/tmp/raw-sar-output.txt$$"
TEMP2="/tmp/sorted-sar-output.txt$$"
CONT=0
sar -d 30 4 > $TEMP1
grep \^"Average" $TEMP1 | awk '{ print $5,$2 }' | sort -k 1n,1 > $TEMP2
if [ -s "$TEMP2" ]
then
   while read file;
   do
      IO=`echo $file | awk '{ print $1 }'`
      TMP_VAL=`echo $file | awk '{ print $1 }'`
      CONT=`expr $CONT + $TMP_VAL`
   echo CONT = $CONT
   done < $TEMP2
   #Echo Result
   echo $CONT
else
   rm -f $TEMP1 $TEMP2 2>/dev/null
   exit 1
fi
rm -f $TEMP1 $TEMP2 2>/dev/null
exit 0

If performance was really important, there is no need for the workfile $TEMP1 and (as Corona688 notes) the maths can be done in awk.

The other issue with your script is possibly the design. Field 5 in "sar" is the Number of Read+Write data transfers PER SECOND. Over the sample period of just 2 minutes you need some decent level of disc activity to get this figure above zero.

Finally the question I should have asked first:
Please explain the statement "execution is too slow and causes problems to the use I need it for".

(You are aware that unix systems can be configured to accumulate "sar" statistics automatically all day every day)?

4

Actually I altered the script to just use one AWK statement as "Corona688" suggested. It now looks like this (also corrected a bug):

#!/bin/ksh
PATH=/usr/bin:/usr/sbin:/sbin; export PATH
TEMP1="/tmp/raw-sar-output.txt$$"
sar -d 5 4 > $TEMP1
AVG_START=`awk '/^Average/ {print NR}' $TEMP1`
TOT_LINES=`cat $TEMP1 |wc -l`
TAIL_LINES=`expr $TOT_LINES - $AVG_START + 1`
tail -$TAIL_LINES $TEMP1 | awk '{ if ($1 == "Average") sum=sum + $5; else sum=sum + $4 } ; END{print sum}'
rm -f "$TEMP1" 2>/dev/null
exit 0

I noticed afterwards that the sar command was causing most of the delay but then again I am under the impression that bash was a bit faster.

To answer "methyl"'s question, I will use this script along with net-SNMP extend (or exec if you will) functionality. The idea is to graph the accumulation of read/writes on servers.

So... Can someone propose a logical interval? The servers that will be queried I assure are very very busy :stuck_out_tongue:

Currently I have it set as

 
sar -d 5 4

this also helps so that my snmpwalks do not time out.

For the sake of learning.. Can someone answer how come the variable scope of the variable "CONT" is not global...?