Variable unexpectedly blank after piped loop

Shell Programming and Scripting
1

Hi,

I was trying out a new way to parse multiple lines from a variable. I liked the "while read" method, so I used echo to pipe the variable into the while loop, the basic structure looking like:

echo $var | while read nextline
 do
    a=blah
done


 echo $a # it is empty!

I am not sure why this is happening. It seems to be the behaviour I'd expect running a subshell, but I didn't think that the pipe could start a subshell.

Any references on why this is happening would be appreciated. I have attached a functional script to this post which demonstrates this issue if you need to see it in action.

Thanks for any assistance.

2

OK you read nextline, where do you use it? (now nextline is a new variable...) Where you trying to get something out of $nextline ?

3

In bash and most shells the while loop the part after the pipe is executed in the background (except ksh and zsh). In those shells, once the part behind the pipe finishes, any variable content will be lost. One can avoid this like this with a code block:

echo "$var" | 
{
  while read nextline
  do
     a=blah
  done
  echo "$a"
}

Or by avoiding the pipe altogether (bash/ksh93/zsh only)

while read nextline
do
   a=blah
done <<< "$var"
echo "$a"
1 Like
4

Thank you very much!

I have chosen to use the above symbol to avoid the pipe, and I've also noted the code block method for use in cases where I need compatibility.

I was slightly confused by the missing dollar sign in your snippet because I thought maybe this triple-less-than is only for variables so the dollar sign is unneeded. I still need to read up on it...

5

You're welcome. Indeed the $-sign went missing! I have added it to my post...

6

Note that as Scrutinizer said, ksh (both ksh88 and ksh93 ) and zsh run the last segment of a pipeline in the current shell execution environment. So, if you were using ksh or zsh instead of bash (or whatever shell you are using), the script you showed us in post #1 in this thread would have produced the output:

blah

from the echo $a command at the end of your script.

7

Hmmm, may I object? Actually, bash also CAN "run the last segment of a pipeline in the current shell execution environment". man bash :

shopt -s lastpipe                     # set option
set +m                                # disable job control
unset a
 echo $var | while read nextline;     # from post#1

   do     a=blah;
   done


echo $a                               # it is empty!
blah                                  # it's NOT!
1 Like
8

Hi RudiC,
You may indeed object. But, you need a very recent version of bash to get that shopt option. On the latest release of macOS, the bash installed is version 3.2.57 which produces the following when trying to set that option:

bash-3.2$ shopt -s lastpipe
bash: shopt: lastpipe: invalid shell option name
bash-3.2$

and a man bash | grep lastpipe produces no output.

1 Like
9

Yes - I should have mentioned my bash version: 4.4.19(1)-release