SED inside while loop

UNIX for Beginners Q & A UNIX for Dummies Questions & Answers
1

Hi,

im having problem creating a loop using my code: aside from the fact that the 1st variable (VAR) does not increment, it loops more than the expected output.

for sample purposes, test csv contains 3 lines.

 
#get number of lines in the file
 lines=$( wc -l < test.csv )
 
  VAR=1
  VAR2=1
 while [ $VAR -le $lines+1 ]
 do
 while [ $VAR2 -le $lines+1 ]
 do
      sed "s/,,/,one"$VAR"too"$VAR2",/" test.csv > test1.csv 
      VAR2=$(( $VAR2 + 1 ))
  done
 
  VAR=$(( $VAR + 1 ))
 
 done

Output of the code is like this, this is caused by the sed command, its like the script executes first the sed command then the loop:

 
a,b,c,d,e,one1too1,f
a,b,c,d,e,one1too1,f
a,b,c,d,e,one1too1,f
a,b,c,d,e,one1too2,f
a,b,c,d,e,one1too2,f
a,b,c,d,e,one1too2,f
a,b,c,d,e,one1too3,f
a,b,c,d,e,one1too3,f
a,b,c,d,e,one1too3,f
a,b,c,d,e,one1too4,f
a,b,c,d,e,one1too4,f
a,b,c,d,e,one1too4,f

But the output should be:

a,b,c,d,e,one1too1,f
a,b,c,d,e,one1too2,f
a,b,c,d,e,one1too3,f
a,b,c,d,e,one1too4,f

i hope i explained and posted this question appropriately...

Hope someone could help..really appreaciate it.. thanks!

2

I'm still not clear on what you are after here, but a couple of obvious things:

 lines=$( wc -l < test.csv )
 lines=$((lines + 1))
 
  VAR=1
  VAR2=1

 while [ $VAR -le $lines  ]
 do
 while [ $VAR2 -le $lines  ]
 do
      sed "s/,,/,one"$VAR"too"$VAR2",/" test.csv  > test1.csv
      VAR2=$(( VAR2 + 1 ))
  done
 
  VAR=$(( $VAR + 1 ))
 
 done
3

what i mean is i want to insert incrementing numbers in the varibles $VAR and $VAR2. But the code is making the sed command, which displays all the lines in the file test.csv, $lines number of times before it loops and repeats again.

that is why the result is

 
a,b,c,d,e,one1too1,f
a,b,c,d,e,one1too1,f
a,b,c,d,e,one1too1,f
 
a,b,c,d,e,one1too2,f
a,b,c,d,e,one1too2,f
a,b,c,d,e,one1too2,f
 
a,b,c,d,e,one1too3,f
a,b,c,d,e,one1too3,f
a,b,c,d,e,one1too3,f
 
a,b,c,d,e,one1too4,f
a,b,c,d,e,one1too4,f
a,b,c,d,e,one1too4,f

when i tried your sugesstion:

the result looks like this:

 
1
a,b,c,d,e,one1too1,f
a,b,c,d,e,one1too1,f
a,b,c,d,e,one1too1,f
2
a,b,c,d,e,one1too2,f
a,b,c,d,e,one1too2,f
a,b,c,d,e,one1too2,f
3
a,b,c,d,e,one1too3,f
a,b,c,d,e,one1too3,f
a,b,c,d,e,one1too3,f
4
a,b,c,d,e,one1too4,f
a,b,c,d,e,one1too4,f
a,b,c,d,e,one1too4,f
1
2
3
4

what i want the result to be is:

 
a,b,c,d,e,one1too1,f
a,b,c,d,e,one2too2,f
a,b,c,d,e,one3too3,f
a,b,c,d,e,one4too4,f

The word "one" and "too" are fixed and only the numbers (e.g from 1 to 4) should be incrementing.

4

Your inner loop only works once since you do not reset the VAR2 variable. So after the next iterations of the outer loop the inner loop no longer gets executed, VAR2 being equal to 4.
For the output you want you do not need two loops. It could be done with one since var1 and var2 are always the same. Further what you have to understand is that sed operates on the entire input file and will output every line of the input file with the two comma's replaced and write the result to test1.csv. For sed to work only on a specific line, you would need to specify that. Another thing that is going on is that every next loop iteration the output file will get overwritten since you use '>' instead '>>' (append).

So then your loop would look something like this:

# get number of lines in the file
  lines=$( wc -l < test.csv )
 lines=$((lines + 1))
  VAR=1
  while [ $VAR -le $lines ]
  do
     echo sed -n "${VAR}s/,,/,one${VAR}too${VAR},/p"
     sed -n "${VAR}s/,,/,one${VAR}too${VAR},/p" test.csv >> test1.csv
     VAR=$(( $VAR + 1 ))
  done

But the preferred way of going to that file would be like this:

VAR=1
  while read line
  do
     echo "$line"| sed "s/,,/,one${VAR}too${VAR},/"
     VAR=$(( $VAR + 1 ))
  done < test.csv > test1.csv

You could then further optimize by replacing sed with shell variable expansion...

  VAR=1
  while read line
  do
    echo "${line%,,*},one${VAR}too${VAR},${line#*,,}"
    VAR=$(( $VAR + 1 ))
  done < test.csv > test1.csv
5

Hi,
I am not sure how your input file is.
Assuming your input file test.csv as
a,b,c,d,e,,f
a,b,c,d,e,,f
a,b,c,d,e,,f
a,b,c,d,e,,f

the following code will give you the desired output.
cnt=1
while read line; do
sed -n ''$cnt' p' test.csv | sed 's/\,\,/\,one'$cnt'too'$cnt'\,/' >> test1.csv
cnt=$(echo "$cnt + 1" | bc)
done < test.csv

output:
a,b,c,d,e,one1too1,f
a,b,c,d,e,one2too2,f
a,b,c,d,e,one3too3,f
a,b,c,d,e,one4too4,f

Hope this helps.

6

Thank you so much guys for all the help. Now i know why my code was acting like that!! .. this was a great help!!! ... really really appreciated it!