Tune my script

Shell Programming and Scripting
1

Hi !

My script read out data out of 144 files per day - every ten minutes a file with data.
data-file

 
WR030B      306.71       0
WR050B      315.13       0
WR120B      308.34       0
WV030B        3.52        0
WV050B        5.06        0
WV120B        6.65       0
TLUFT02B      8.60        0
TLUFT12B      7.94        0
NSCHLB        0.00         0

my script is working like this for files with the namestructure YYYYMMDDHHMM

 
#!/bin/bash
M=`date +%m`
Y=`date +%Y`
home=$HOME/METE
cd $home
FLIST=`ls $Y$M*`
for i in $FLIST
do
     if test -f $i; then
  Dy=`echo $i | cut -c3-4`
  Dm=`echo $i | cut -c5-6`
  Dd=`echo $i | cut -c7-8`
 UZh=`echo $i | cut -c9-10` 
 UZm=`echo $i | cut -c11-12`
  WV=`grep WV120B $i | awk '{print $2}'`
  WR=`grep WR120B $i | awk '{print $2}'`
  NS=`grep NSCHLB $i | awk '{print $2}'`
T120=`grep TLUFT12B $i | awk '{print $2}'`
# echo $i " --> "$UZh":"$UZm " --> "$Dm"/"$Dd"/"$Dy"-"$UZh":"$UZm "\t" $WV
 printf "$Dm/$Dd/$Dy-$UZh:$UZm\t$WV\t$WR\t$NS\t$T120\n"
 else
 DAT=`date +%Y%m%d`
        printf "$DAT - Datei $i is not here" >> /$HOME/Logfiles/Log_`date +%Y%m%d`.log
    fi
done

so at the end i got one file like this

 
...
05/25/12-03:30 7.28 82.58 0.00 17.29
05/25/12-03:40 7.90 82.08 0.00 17.17
05/25/12-03:50 8.78 78.06 0.00 17.06
05/25/12-04:00 8.73 79.78 0.00 16.88
05/25/12-04:10 8.13 81.56 0.00 16.64
... 

 
time /script.sh

will bring
real 0m41.111s
user 0m1.400s
sys 0m4.134s
So far it is working properly, but i want to speed it up.
How can i change the script to make it more fast?

Thanks in advance!
IMPe

2

Hi

Few points:

  1. The commands with grep and awk combination can be done with awk only, like:
    Instead of
 WV=`grep WV120B $i | awk '{print $2}'`

Use

WV=`awk '/WV120B/{print $2}' $i`

This will get rid of those 4 grep commands.
  1. The date command is used twice in the beginning +"as many files you have" times, whereas just one date would have done the thing for you. also the one inside the loop is not needed since the result is fixed. Instead of these 3 commands, use the below at the very beginning of the script:
      DAT=`date +%Y%m%d`
      Y=`echo ${DAT:0:4}`
      M=`echo ${DAT:4:2}`

This step will get rid of your umpteen date commands.

  1. Instead of the statement
Dy=`echo $i | cut -c3-4`

Use:

 Dy=` echo ${i:2:2}`

This will get rid of those 4 cut commands.
  1. In place of the command:
FLIST=`ls $Y$M*`

Use:

 FLIST=`echo $Y$M*`

ls is an external command.

Guru.

1 Like
3

Like guruprasadr suggests, it is important to limit the number of external programs as much as possible, especially within the loop. Try:

#!/bin/bash
fpat=$(date +%Y%m)*
home=$HOME/METE
cd $home
for i in $fpat
do
  if test -f "$i"
  then
    Dy=${i:2:2} Dm=${i:4:2} Dd=${i:6:2}
    UZh=${i:8:2} UZm=${i:10:2}
    res=$(awk '{A[$1]=$2}END{print A["WV120B"], A["WR120B"], A["NSCHLB"], A["TLUFT12B"]}' OFS='\t' "$i")
    printf "%s/%s/%s-%s:%s\t%s\n" $Dm $Dd $Dy $UZh $UZm "$res"
  else
    DAT=$(date +%Y%m%d)
    printf "$DAT - Datei $i is not here" >> "$HOME/Logfiles/Log_$(date +%Y%m%d).log"
  fi
done

--
Guru: these suggestions could be further improved by leaving out some of the echo + back tick combinations.

1 Like
4

Thanks a lot!
With the advancement of @guruprasadpr the script takes only half the time, bud with @Scrutinizer improvement it only needs 5.4 sec.
really great!! :b:

IMPe