Bash Variable scope - while loop while reading from a file

Cope sample1: test.sh

i=0
echo " Outside loop i = $i "
while [ $i -le 5 ]
do
i=$(( $i + 1))
echo "Inside loop i = $i "
done
echo " Out of loop i is : $i "

When run output :

Outside loop i = 0
Inside loop i = 1
Inside loop i = 2
Inside loop i = 3
Inside loop i = 4
Inside loop i = 5
Inside loop i = 6
 Out of loop i is : 6

Code sample 2: test_1.sh

cat file1
AAAAAAAAAAA
BBBBBBBBBBB
CCCCCCCCCCC
DDDDDDDDDDD
EEEEEEEEEEE
i=0
echo " Outside loop i = $i "
cat file1 |while read line
do
i=$(( $i + 1))
echo "Inside loop i = $i "
done
echo " Out of loop i is : $i "

When run ouputput:

test_1.sh

 Outside loop i = 0
Inside loop i = 1
Inside loop i = 2
Inside loop i = 3
Inside loop i = 4
Inside loop i = 5
 Out of loop i is : 0

I'm puzzled.

In the second script (test_1.sh), I expect the output of the variable i to be 5 not 0.

Can someone help me understand why it is back to 0 ( why not 5?)

Hello Adarshreddy01,

IMHO that's because you are passing output of cat file1 command to while loop which is considered as a new shell(or in other language creates a sub shell to execute while loop). So once while loop is done with execution its new value of i gets lost and when you try to print its value; it prints OLD value for i .

Now lets do this with another way(Without using cat file | while.....

cat file.ksh
i=10
echo " Outside loop i = $i "
while read line
do
i=$(( $i + 1))
echo "Inside loop i = $i "
done < "file1"
echo " Out of loop i is : $i "

When you run script file.ksh we will see following.

 Outside loop i = 10
Inside loop i = 11
Inside loop i = 12
Inside loop i = 13
Inside loop i = 14
Inside loop i = 15
 Out of loop i is : 15

I believe that is proved why in your attempt i's value is getting lost.

Thanks,
R. Singh

2 Likes

It's good to mention that there is a major difference between bash and ksh running the same code.
Which is expected due to ksh / bash difference in implementation.

# ./in_ksh.sh
 Outside loop i = 0
Inside loop i = 1
Inside loop i = 2
Inside loop i = 3
Inside loop i = 4
Inside loop i = 5
 Out of loop i is : 5

./in_bash.sh
 Outside loop i = 0
Inside loop i = 1
Inside loop i = 2
Inside loop i = 3
Inside loop i = 4
Inside loop i = 5
 Out of loop i is : 0

Regards
Peasant.

2 Likes

As Ravinder has already said it is because of the pipeline. Basically in classical Bourne Shell (and its descendants, bash - Bourne Again Shell - among them) code like this:

process1 | process2

is executed by running the process1 in the main shell and process2 in its own subshell. Therefore the while-loop you use runs in its own subshell, with its own variables and inside this shell your variables work - which is why your counter is increased - but when the subshell is left (at the end of the loop) all these variables are deleted. This is why your counter variable is zero at the end.

Notice that the other descendant of the Bourne shell - the Korn Shell, ksh - does things differently. In ksh your code would work as you obviously expect because it executes the last element of a pipeline in the main shell. You can force bash (since, IIRC version 4) to do the same but you must set this option explicitly:

shopt -s lastpipe

will set the behavior of bash to exactly the same as ksh always had. So your options are: switch to ksh (most of the code is the same but there are subtle differences between bash and ksh) or set the lastpipe option.

By the way: notice that pipelines and redirections word differently in bash, regardless of the lastpipe-setting! This simple line-counter:

(( lines = 0 ))
cat /path/to/file | while read line ; do
     (( lines++ ))
done
echo $lines

will not work (for the same reasons as your code) but:

(( lines = 0 ))
while read line ; do
     (( lines++ ))
done < /path/to/file
echo $lines

will.

I hope this helps.

bakunin

/PS: only now saw that peasant has also addressed the ksh<->bash difference. He is absolutely right.

3 Likes

As is often the case with a little bit of jiggery pokery there is a pseudo-back-door.
The storage device can be your friend although it will slow things down with the disk thrashing and huge numbers.
NOTE: This uses 'dash' which means it is fully POSIX compliant.

#!/usr/local/bin/dash

i=0
echo "Outside loop i = ${i}."
cat self.txt | while read -r line
do
    # Read the line, there are 5 lines...
    i=$(( i + 1 ))
    echo "Line ${i}."
    echo "${i}" > i
done
i=$( cat i )
echo "Out of loop i = ${i}."

Results OSX 10.14.3, default terminal.

Last login: Tue Jul 30 14:28:01 on ttys000
AMIGA:amiga~> cd Desktop/Code/Shell
AMIGA:amiga~/Desktop/Code/Shell> chmod 755 subshell.sh
AMIGA:amiga~/Desktop/Code/Shell> ./subshell.sh
Outside loop i = 0.
Line 1.
Line 2.
Line 3.
Line 4.
Line 5.
Out of loop i = 5.
AMIGA:amiga~/Desktop/Code/Shell> _
1 Like

This is especially for you:

#! /bin/bash

pInitPermStor ()
{
if [ ! -d "$fStorage" ] ; then
     return $(mkdir -p "$fStorage")            # create storage dir if not there
fi

return $(rm -f "$fStorage/*")                  # empty it if it already exists
}




pRemovePermStor ()
{
if [ ! -d "$fStorage" ] ; then
     return 1                                  # shouldn't happen
fi

return $(rm -f "$fStorage")                    # clean up
}




pGetVar ()
{
local chVar="$1"

if [ -f "${fStorage}/${chVar}" ] ; then
     printf "%s=%s\n" "$chVar" $( < "${fStorage}/${chVar}" )
else
     printf "$chVar"=\"\"
fi

return 0
}


pPutVar ()
{
local chVar="$1"
local chValue="$2"

if ! printf "%s\n" "$2" > "${fStorage}/${chVar}" ; then
     return 1
fi

return 0
}


# ------------------ main()
declare fStorage="/path/to/dir/$$"

if ! pInitPermStor ; then
     printf "ERROR: cannot initialise permanent variable storage\n" >&2
     exit 1
fi

pPutVar "x" "abcd"             # set value of variable x to "abcd"
eval $( pGetVar "x" )          # retrieve and set variable x
echo var x is: $x

if ! pRemovePermStor ; then
     printf "ERROR: permanent variable storage not found\n" >&2
     exit 1
fi
exit 0

ROFL, i doubt this helps anyone but it was fun to write.

bakunin

/edited following MadeInGermanys suggestion

bakunin, I suggest to once and globally define the variable

fStorage="/path/to/dir/$$"

.
because some shells run a sub shell in a separate process, and $$ changes.

You are right. I have edited the code above.

bakunin

Hi bakunin...

Ha ha, but where is 'i'?
The OP wanted 'i'... <wink>