Would love to get some quick help on below requirement.
I am trying to process mpstat output from multiple blades of my server
I would like to assign this the output to an array and then use it for post processing. How can I use a two dimensional array and assign these value
You failed to mention the shell you use. In case it's bash , try "associative arrays":
declare -A cpuusage
{ read
read
read -a HD
while read -a TMP
do for i in ${!HD[@]}
do cpuusage[${TMP[1]}","${HD[$i]}]=${TMP[$i]}
done
done
} < file
echo ${cpuusage[2,%soft]}
0.04
I wonder if your idea makes sense at all. According to man mpstat the reported values are average since the system boot - unless you give an interval.
And if you want the average over all CPUs, consider vmstat or iostat -c (again, with an interval).
Associative arrays are as of bash 4 . If that is not available on your system, you could try ksh93 or zsh with a bit of syntax adjustment ( typeset instead of declare , read -A instead of read -a , ...)
Now the script works with your recommended changes
I do have a question though. The script we have drafted feeds input from a file.
However how could I directly feed the output of
mpstat
command into the script
Will this work ?
#!/bin/ksh93
typeset -A cpuusage
{ read
read
read -A HD
while read -A TMP
do for i in ${!HD[@]}
do cpuusage[${TMP[1]}","${HD[$i]}]=${TMP[$i]}
done
done
} < `mpstat -P ALL`
echo ${cpuusage["14,%soft"]}
Lets say if I run this script every 5mins, Is it possible to increment the values of cpuusage array to add to its previous value and store it . Then I can easily average it based on the number of sampling at the final sampling period ?
No. The operand to the redirection operator (i.e. < ) is the name of a file to be opened for reading. The output from running the mpstat utility is not the pathname of a file. You could, however, change:
} < `mpstat -P ALL
to:
} <<< `mpstat -P ALL`
or, using the non-obsolete form of command substitution:
} <<< $(mpstat -P ALL)
Of course. Arithmetic expansions work just fine as the right hand side of an assignment command:
As you can see, when we store the CPU usage in cpuusage array, I only need to keep the CPU usage accumulated (add the reading to the previous reading). You suggestion to add using below command even tries to add first column and CPU#, which is not required
It always helps to experiment and do some reading, e.g. man pages . For your lastmost problem, start the evaluation/calculation only at the third column. Try
{ read
read
read -a HD
while read -a TMP
do for (( i=2; i<${#HD[@]}; i++ ))
do ((cpuusage[${TMP[1]}","${HD[$i]}]+=${TMP[$i]}))
done
done
} < file
Please note that this is tested in bash using integers only as bash unlike ksh doesn't offer shell floating number arithmetics.
I assumed that you had written the code you had shown us and understood what it was doing. I showed you a syntax for adding fields and assumed that you would be able to modify your own code to do straight assignments or to skip assigning anything at all for data in columns you didn't want to sum.
Assuming that you don't need to keep any data from the first two columns of your data, the code RudiC suggested above should also work just fine in a recent Korn shell as long as you change the read -a to read -A in both of the places where it occurs.
I was able to modify the script and able to get the desired behavior. Below is my new script
#!/bin/ksh93
typeset -A cpuusage
{ read -A HD
while read -A TMP
do for i in ${!HD[@]}
do
cpuusage[${TMP[0]}","${HD[$i]}]=$(( ${cpuusage[${TMP[0]}","${HD[$i]}]} + ${TMP[$i]} ))
done
done
} <<< $(mpstat -P ALL | awk '{ $1=""; print $0 }' | awk 'NR>=3')
mpstat -P ALL | awk '{ $1=""; print $0 }' | awk 'NR>=3'
echo ${cpuusage["14,%sys"]}
Now moving to my next step.
Question:
I am using this cpuusage array within a while loop and repeating it for a pre-defined period. Now I want to use this cpuusage array and extract values outside the while loop and save it into a file. Looks like the scope of cpuusage is not available outside the while loop. Below is the snippet of my main script
while true
do
echo "Sampling started"
typeset -A cpuusage
{ read -A HD
while read -A TMP
do for i in ${!HD[@]}
do
cpuusage[${TMP[0]}","${HD[$i]}]=$(( ${cpuusage[${TMP[0]}","${HD[$i]}]} + ${TMP[$i]} ))
done
done
} <<< $(mpstat -P ALL | awk '{ $1=""; print $0 }' | awk 'NR>=3')
echo "sampling done"
if [ $SCANTIME -eq $GRANULARITYPERIOD ]
then
break 2
fi
sleep $SCANTIME
SCANTIME=$((SCANTIME+5))
echo "------------------"
echo "SCANNING TIME: $SCANTIME"
done
echo "TRYING TO PROCESS OUTPUT"
for CPUCORE in [0..19]
do
cat >> $pmFilename <- EOM
<r p="1">`echo "scale=2; ${cpuusage["$CPUCORE,%iowait"]}/3" | bc -l | sed 's/^\./0./'`</r>
<r p="2">`echo "scale=2; ${cpuusage["$CPUCORE,%irq"]}/3" | bc -l | sed 's/^\./0./'`</r>
<r p="3">`echo "scale=2; ${cpuusage["$CPUCORE,%nice"]}/3" | bc -l | sed 's/^\./0./'`</r>
<r p="4">`echo "scale=2; ${cpuusage["$CPUCORE,%soft"]}/3" | bc -l | sed 's/^\./0./'`</r>
<r p="5">`echo "scale=2; ${cpuusage["$CPUCORE,%sys"]}/3" | bc -l | sed 's/^\./0./'`</r>
<r p="6">`echo "scale=2; ${cpuusage["$CPUCORE,%idle"]}/3" | bc -l | sed 's/^\./0./'`</r>
<r p="7">`echo "scale=2; ${cpuusage["$CPUCORE,%usr"]}/3" | bc -l | sed 's/^\./0./'`</r>
EOM
done
---------- Post updated 11-09-16 at 12:35 AM ---------- Previous update was 11-08-16 at 05:49 PM ----------
RudiC - Not really. You have seen how good I am in the scripting
Attempting this would be another big exercise. I will look at it once I finish this script
MadeInGermany - Thanks for the recommendations, I will adopt your changes
------
Another question related to how to initiate execution of this script that I am preparing
I do not want to use cronjob as we do not have permission to use
I would like to start the script on the next possible quarter hour
e.g. Current time is 11:46. If I run the script, it should pause initially and continue executing from 12:00
