Tuning function

Shell Programming and Scripting
1

This is my function in UNIX file. In this function I am

-> first replacing spaces in character 19-27 with 0
-> then if it's all zeros ( 9 zeros) replace it with space

The reason I have to make it to 0 first is that my requirement is that if this field is having value of 0 , replace it with spaces. The zeros in coming in various ways like-

All zeros. (000000000)
Spaces in between. (0000 0)
Only 1 zero.( 0)

This function is taking 2 minutes 25 seconds for 2000 records which is very very slow. We can have time window of 25-30 minutes and we can get upto 50000 records in1 day. Could someone suggest some ways to tune this fucntion.

modiify_customer_feed_id()
{
   while IFS='' read line
   do
      f1="$(echo "$line" | cut -c1-18)"
      f2="$(echo "$line" | cut -c19-27)"
      f3="$(echo "$line" | cut -c28-854)"
      f2=$(echo "$f2"|sed 's/ /0/g')
      printf "%s%s%s\n" "$f1" "$f2" "$f3"
   done < ${SOURCEFILE} > $id_interim_file
   echo " ID PROCESS 1 "
   while IFS='' read line
   do
      f1="$(echo "$line" | cut -c1-18)"
      f2="$(echo "$line" | cut -c19-27)"
      f3="$(echo "$line" | cut -c28-854)"
      f2=$(echo "$f2"|sed 's/000000000/ /g')
      printf "%s%s%s\n" "$f1" "$f2" "$f3"
   done < $id_interim_file > $id_final_file
   echo " ID PROCESS 2 "
   if [[ $? -ne 0 ]]
   then
      rm -f $id_interim_file $id_final_file
      return 1
   else
      mv -f $id_final_file $SOURCEFILE
      rm -f $id_interim_file
      return 0
   fi
}
2

try this:

echo 'some_data_blablah_0       00' | awk '{
b=substr($0,1,18)
s=substr($0,19,9)
e=substr($0,28,length($0)-27)
gsub (" ","0",s)
if (s ~ /0{9}/) {gsub (" ","0",s)}
print b""s""e}'

not tested yet.

3

This is my final function. For 2000 records running in 3 minutes. Can this be tuned . Many Thanks in advance.

modify_customer_feed()
{
while IFS='' read line
do
f1="$(echo "$line" | cut -c1-18)"
f2="$(echo "$line" | cut -c19-27)"
f2=$(echo "$f2"|sed 's/ /0/g')
f2=$(echo "$f2"|sed 's/000000000/         /g')
f3="$(echo "$line" | cut -c28-686)"
f4="$(echo "$line" | cut -c687-702)"
f4=$(echo "$f4"|sed 's/ /0/g')
f4=$(echo "$f4"|sed 's/0000000000000000/                /g')

f5="$(echo "$line" | cut -c703-718)"
f5=$(echo "$f5"|sed 's/ /0/g')
f5=$(echo "$f5"|sed 's/0000000000000000/                /g')
f6="$(echo "$line" | cut -c719-854)"
printf "%s%s%s%s%s%s\n" "$f1" "$f2" "$f3" "$f4" "$f5" "$f6"
done < ${SOURCEFILE}    >  $id_final_file

if [[ $? -ne 0 ]]
then
  rm -f  $id_final_file
  return 1
else
  mv -f $id_final_file $SOURCEFILE
  return 0
fi
}
4

Are you using bash?

5

you are having lots of redirection and command substitution on each line which makes this bulky.

If you can try the awk solution, that would be faster.

with some correction and removed typos,

awk '{
b=substr($0,1,18)
s=substr($0,19,9)
e=substr($0,28,length($0)-27)
gsub (" ","0",s)
if (s ~ /000000000/) {gsub ("0"," ",s);print b s e} else {print $0}}' file