Hi every one, im kind new in scripting.
i need to write a script that will collect my storage usage (df -k) like every hour and save the max amout and min amout into csv file so over time lest say one month we have the min/max values. Is it possible with awk command ?
Any ideas are welcome.
Welcome venus699,
I have a few to questions pose in response first:-
- What have you tried so far?
- What output/errors do you get?
- What OS and version are you using?
- What are your preferred tools? (C, shell, perl, awk, etc.)
- What logical process have you considered? (to help steer us to follow what you are trying to achieve)
Most importantly, What have you tried so far?
There are probably many ways to achieve most tasks, so giving us an idea of your style and thoughts will help us guide you to an answer most suitable to you so you can adjust it to suit your needs in future.
We're all here to learn and getting the relevant information will help us all.
Kind regards,
Robin
actualy just tryed some awk comands. and did some research around it, but can๏ฟฝt find anything that compare 2 files. lest say:
- Record the first file
df -kP > file1.csv
- compare file1 to max.csv and save the respective values that are greater than the previous we had on max.csv
I'm open to ideas, I'm trying to use awk maybe python but I have low skills.
I'm using opensuse.
---------- Post updated 09-13-17 at 08:26 AM ---------- Previous update was 09-12-17 at 09:52 AM ----------
This is what I have archive atm, still not done yet having some troubles with duplicate fields for some reason. if any one up to test it would be cool.
fileactual="results/fileactual.csv"
fileMaxMin="results/maxminconsumption.csv"
filemax="results/filemax.csv"
filemin="results/filemin.csv"
filecomparemax="results/comparemax.csv"
filecomparemax2="results/compare2max.csv"
filecomparemin="results/comparemin.csv"
filecomparemin2="results/compar2min.csv"
#--------print all the df data
df -kP | awk '{print $0}' > $fileactual
if [ ! -f $filemax ]; then
df -kP | awk '{print $0}' > $filemax
fi
if [ ! -f $filemin ]; then
df -kP | awk '{print $0}' > $filemin
fi
#--------past the 2 files so later we can compare
paste -d" " $fileactual $filemax > $filecomparemax
paste -d" " $fileactual $filemin > $filecomparemin
#--------remove first line from DF command.
awk '{if (NR!=1) {print}}' $filecomparemax > $filecomparemax2
awk '{if (NR!=1) {print}}' $filecomparemin > $filecomparemin2
#-------selecing the biger value on column 3 = used space and print the row
awk '$3 <= $9 {print $1,$2,$3,$4,$5,$6}; $3 >= $9 {print $7,$8,$9,$10,$11,$12}' $filecomparemax2 > $filemax
#-------selecing the lower value on column 3 = used space and print the row
awk '$3 >= $9 {print $1,$2,$3,$4,$5,$6}; $3 <= $9 {print $7,$8,$9,$10,$11,$12}' $filecomparemin2 > $filemin
echo 'THIS IS THE MAX VALUES
------------------------------' > $fileMaxMin
awk 'NR==1' $fileactual >> $fileMaxMin
cat $filemax >> $fileMaxMin
echo '-----------------------------
THIS IS THE LOWER VALUES
------------------------------' >> $fileMaxMin
awk 'NR==1' $fileactual >> $fileMaxMin
cat $filemin >> $fileMaxMin
rm $filecomparemax
rm $filecomparemax2
rm $filecomparemin
rm $filecomparemin2
rm $fileactual
probably not the best approach looking for new ideas.
i made it work, does the job i needed.
fell free to use it, dont forget when opening the file maxminconsumption.csv to import it into excel,i use -> , to discting each columm.
Cheers
#|/bin/bash
# ----------------------------------------------- #
# storageconsumption.sh #
# Storage max / min values #
# Version 1.0 #
# by: #
# ----------------------------------------------- #
fileactual="fileactual.csv"
fileMaxMin="maxminconsumption.csv"
filemax="filemax.csv"
filemin="filemin.csv"
filecomparemax="comparemax.csv"
filecomparemin="comparemin.csv"
#--------print all the df data to the actualfile
df -hP | awk '{print $0}' > $fileactual
#-------Check if file exists, if it donst will save fresh data from DF command
if [ ! -f $filemax ]; then
df -hP | awk '{print $0}' > $filemax
fi
if [ ! -f $filemin ]; then
df -hP | awk '{print $0}' > $filemin
fi
#--------past the 2 files in a new one so in the next command "awk" we can compare the columms
paste -d" " $fileactual $filemax > $filecomparemax
paste -d" " $fileactual $filemin > $filecomparemin
#-------selecing the biger value on column 3 = used space and print the total row. colummns 1 to 6 is from the actual file, 7 to 12 is from the filemax / filemin
awk '$3 > $9 {print $1","$2","$3","$4","$5","$6}; $3 < $9 {print $7","$8","$9","$10","$11","$12} ; $3==$9 {print $7","$8","$9","$10","$11","$12}' $filecomparemax > $filemax
#-------selecing the lower value on column 3 = used space and print the row
awk '$3 < $9 {print $1","$2","$3","$4","$5","$6}; $3 > $9 {print $7","$8","$9","$10","$11","$12} ; $3==$9 {print $7","$8","$9","$10","$11","$12}' $filecomparemin > $filemin
#-------print the header of our final file.
echo 'THIS IS THE MAX VALUES
-----------------,-----------------,-----------------,-----------------,-----------------,----------------- ' > $fileMaxMin
#-------print our MAX values
cat $filemax >> $fileMaxMin
echo '-----------------,-----------------,-----------------,-----------------,-----------------,-----------------
THIS IS THE LOWER VALUES
-----------------,-----------------,-----------------,-----------------,-----------------,----------------- ' >> $fileMaxMin
#-------print our MIN values
cat $filemin >> $fileMaxMin
#------- delete desnecessary files.
rm $filecomparemax
rm $filecomparemin
rm $fileactual
1 Like
i did another one, to collect data over time, the space of folder in filepath.
fell free to use it
also if any one know how to improve the code to be more efficient just share, im here to learn
filepath="/home/"
foldername="*"
data="data1.csv"
actualdata2="data2.csv"
actualdata3="data3.csv"
actualheader="data4.csv"
#---------------new file if $data dont exist yet
if [ ! -e $data ]; then
echo " " >> $data
for file in $filepath$foldername
do
du -sh $file | awk '{print $2}' >> $data
done
fi
#---------------input from wc -l so later we can check if we need to update our file
countfilepath=`ls $filepath | wc -l`
countfileline=`awk '{if (NR!=1) {print}}' data1.csv | wc -l`
#------------- file existe do this.
if [ -e $data ]; then
#--------------- if number of files is not equal to number of lines in file
if [ $countfilepath -eq $countfileline ]; then
echo $(date +%m'-'%d) >> $actualdata2
for file in $filepath$foldername
do
du -sh $file | awk '{print $1}' >> $actualdata2
done
paste -d" " $data $actualdata2 > $actualdata3
mv $actualdata3 $data
rm $actualdata2
else
mv $data $(date +%Y%m%d)'_'$(date +%H%M%S)'_'$data
echo " " >> $data
for file in $filepath$foldername
do
du -sh $file | awk '{print $2}' >> $data
done
echo $(date +%m'-'%d) >> $actualdata2
for file in $filepath$foldername
do
du -sh $file | awk '{print $1}' >> $actualdata2
done
paste -d" " $data $actualdata2 > $actualdata3
mv $actualdata3 $data
rm $actualdata2
fi
fi