String concatenation not working in a loop

Shell Programming and Scripting
1

Hi,

First post, so I hope someone can help me with this weirdness :slight_smile:

I have a number files with some rows of information I want to extract, at the same time I want to add to a string some details from the file. I have found two different ways of looping over rows in a file, but one method doesn't let me concatenarte strings and I don't know why....

The first method sets the IFS environmental variable. I don't really favour this method, but it works..

#!/bin/bash -
OLDIFS=$IFS
IFS='
'

STE="cat"
  
for FILE in $( ls *-df | cut -f 4 -d / ); do
  N=0
  for LINE in $( cat $FILE ); do
    N=$((N+1))
    STE="$STE""dog"
  done
done
echo "$STE" && exit

and produces:
catdogdogdogdogdogdogdogdogdogdogdogdogdogdogdogdo gdogdogdogdogdogdogdogdogdogdogdogdog

Method 2 uses "read" but prevents the concatenation of the string:

#!/bin/bash -
STE="cat"
  
for FILE in $( ls *-df | cut -f 4 -d / ); do
  N=0
  cat "$FILE" | while read LINE ; do
    N=$((N+1))
    STE="$STE""dog"
  done
done
echo "$STE" && exit

and produces:
cat

Why is there a difference in the two methods? I would rather not be changing environmental variables and a simple loop is my preferred choice.

Thanks for your help, as this has been driving me crazy...
Stephen

2

The difference lies in the use of a pipe if you are using bash. This would work in ksh

cat "$FILE" | while read LINE ; do
    N=$((N+1))
    STE="$STE""dog"
  done

In bash (contrary to ksh) the last command in the pipe does not run in the current process so the variable STE loses its value once it finishes.

S.

3

So if I want to use BASH (the company standard), the only option is to change the IFS variable? And then change it back again?

Thanks,
LITW

4

As Scrutinizer mentioned, with bash the commands after the pipe run in a subshell and variables in a subshell are not visible outside the block of code in the subshell and they are not accessible to the parent process.

You can avoid the use of a pipe (and useless use of cat) like:

STE="cat"
  
for FILE in $( ls *-df | cut -f 4 -d / ); do
  N=0
  while read LINE ; do
    N=$((N+1))
    STE="$STE""dog"
  done < "$FILE"
done
echo "$STE" && exit
5

Thanks to you both, I am really glad I joined this forum now :slight_smile:

Works like a charm. A bit of a follow up question though, is it bad practise to change the IFS variable like in the first example?

Thanks again,
LITW

6

If you don't read lines (filenames) with spaces it isn't necessary to change the IFS.

Regards