Help with While loop using if else

Shell Programming and Scripting
1

Hi ,
I am executing a while loop .The only condition is , as far as the cumulative sum of the 2nd column from each line passed from the txt file reaches value of 12 , echo those lines .But once the cumulative sum reaches 12 , restart the loop setting the cumulative sum variable to 0 . But in the below snapshot , the cumulative sum is getting set to 0 only at the line 5 instead of 4 . The line 4 is skipped due to this reason during the loop iteration. This is also true at line 7 which also getting skipped as the cumulative sum variable is getting reset to 0 at line 8 in the subsequent iteration. Please help with any suggestions

source file : file_size_num.txt

1,3,/informatica/apps/logs/fin/ote/TgtFiles/Destination/CWXX/CW1900
2,4,/informatica/apps/logs/fin/ote/TgtFiles/Destination/CWXX/CW1901
3,5,/informatica/apps/logs/fin/ote/TgtFiles/Destination/CWXX/CW1901
4,1,/informatica/apps/logs/fin/ote/TgtFiles/Destination/CWXX/CW1901
5,2,/informatica/apps/logs/fin/ote/TgtFiles/Destination/CWXX/CW1900
6,6,/informatica/apps/logs/fin/ote/TgtFiles/Destination/CWXX/CW1901
7,7,/informatica/apps/logs/fin/ote/TgtFiles/Destination/CWXX/CW1901
8,8,/informatica/apps/logs/fin/ote/TgtFiles/Destination/CWXX/CW1901

#!/bin/sh
sum=0
while read line
 do
 var1="$(echo "$line" | awk -F "," '{print $2}')"
 sum=$(expr "$sum" + "$var1")
 echo $sum 
 if [ $sum -le 12 ]
 then
 echo $line
 else
 sum=0
line=$line
 fi
done < file_size_num.txt

output :
in the below snapshot , the cumulative sum is getting set to 0 only at the line 5 instead of 4 . The line 4 is skipped due to this reason during the loop iteration. This is also true at line 7 which also getting skipped as the cumulative sum variable is getting reset to 0 at line 8 in the subsequent iteration.

-bash-4.2$ sh bash.sh
3
1,3,/informatica/apps/logs/fin/ote/TgtFiles/Destination/CWXX/CW1900
7
2,4,/informatica/apps/logs/fin/ote/TgtFiles/Destination/CWXX/CW1901
12
3,5,/informatica/apps/logs/fin/ote/TgtFiles/Destination/CWXX/CW1901
13
2
5,2,/informatica/apps/logs/fin/ote/TgtFiles/Destination/CWXX/CW1900
8
6,6,/informatica/apps/logs/fin/ote/TgtFiles/Destination/CWXX/CW1901
15
8
8,8,/informatica/apps/logs/fin/ote/TgtFiles/Destination/CWXX/CW1901
-bash-4.2$
2

Try this:

$ awk -F, 'sum < 12 {print; print sum+=$2; next} {print "RESET"; sum=$2; print; print sum}' infile
1,3,/informatica/apps/logs/fin/ote/TgtFiles/Destination/CWXX/CW1900
3
2,4,/informatica/apps/logs/fin/ote/TgtFiles/Destination/CWXX/CW1901
7
3,5,/informatica/apps/logs/fin/ote/TgtFiles/Destination/CWXX/CW1901
12
RESET
4,1,/informatica/apps/logs/fin/ote/TgtFiles/Destination/CWXX/CW1901
1
5,2,/informatica/apps/logs/fin/ote/TgtFiles/Destination/CWXX/CW1900
3
6,6,/informatica/apps/logs/fin/ote/TgtFiles/Destination/CWXX/CW1901
9
7,7,/informatica/apps/logs/fin/ote/TgtFiles/Destination/CWXX/CW1901
16
RESET
8,8,/informatica/apps/logs/fin/ote/TgtFiles/Destination/CWXX/CW1901
8
3

Not sure I fully understand what you're after (please apply some care when phrasing your request), and infering from your prompt you have bash , try

while IFS="," read F1 F2 F3; do ((sum+=F2)); if [ $sum -le 12 ]; then echo $F1,$F2,$F3; else sum=0; fi; done < file
1,3,/informatica/apps/logs/fin/ote/TgtFiles/Destination/CWXX/CW1900
2,4,/informatica/apps/logs/fin/ote/TgtFiles/Destination/CWXX/CW1901
3,5,/informatica/apps/logs/fin/ote/TgtFiles/Destination/CWXX/CW1901
5,2,/informatica/apps/logs/fin/ote/TgtFiles/Destination/CWXX/CW1900
6,6,/informatica/apps/logs/fin/ote/TgtFiles/Destination/CWXX/CW1901
8,8,/informatica/apps/logs/fin/ote/TgtFiles/Destination/CWXX/CW1901
4
Moderator comments were removed during original forum migration.
5

Yes, because what you say you want and what you have written isn't consistent i have take the liberty to annotate your code:

#!/bin/sh
sum=0
while read line
 do
 var1="$(echo "$line" | awk -F "," '{print $2}')"   
 sum=$(expr "$sum" + "$var1")                       
 echo $sum 
 if [ $sum -le 12 ]                                 # if sum is less or equal 12
 then
 echo $line                                         # then - echo the line
 else
 sum=0                                              # else - reset the sum
line=$line
 fi
done < file_size_num.txt

Notice, that you need to reset the sum not only if it is greater than 12 (this is the effective logic behind the else-part of "-le") but also when 12 exactly is reached. That means:

#!/bin/sh
sum=0
while read line
 do
 var1="$(echo "$line" | awk -F "," '{print $2}')"
 sum=$(expr "$sum" + "$var1")
 echo $sum 
 if [ $sum -le 12 ] ; then
     echo $line
 fi
 if [ $sum -ge 12 ] ; then
     sum=0
 fi
done < file_size_num.txt

A few words about the rest of your code:

sum=$(expr "$sum" + "$var1")

This is Bourne-shell-syntax and you shouldn't use it when using bash. bash can handle integers quite well:

(( sum = sum + var1 ))

The way you extract the value from the line is also not optimal: instead of using an external program (which is costly in terms of resources) do it directly in the shell. Instead of:

while read line ; do
    var1="$(echo "$line" | awk -F "," '{print $2}')"
done < file_size_num.txt

Do it like this:

while read line ; do
    echo "$line" | IFS=, read junk var1 junk
    echo "$line"
    echo "$var1"
done < file_size_num.txt

I hope this helps.

bakunin

6

zaxxon . You are close :slight_smile:

Hi all ,
The below is the expected result . Reset at every point the cumulative sum becomes greater than 12 .

1,3,/informatica/apps/logs/fin/ote/TgtFiles/Destination/CWXX/CW1900
3
2,4,/informatica/apps/logs/fin/ote/TgtFiles/Destination/CWXX/CW1901
7
3,5,/informatica/apps/logs/fin/ote/TgtFiles/Destination/CWXX/CW1901
12
RESET
4,1,/informatica/apps/logs/fin/ote/TgtFiles/Destination/CWXX/CW1901
1
5,2,/informatica/apps/logs/fin/ote/TgtFiles/Destination/CWXX/CW1900
3
6,6,/informatica/apps/logs/fin/ote/TgtFiles/Destination/CWXX/CW1901
9
RESET
7,7,/informatica/apps/logs/fin/ote/TgtFiles/Destination/CWXX/CW1901
7
RESET
8,8,/informatica/apps/logs/fin/ote/TgtFiles/Destination/CWXX/CW1901
8
7

Please be aware that zaxxon's proposal resets at "greater than AND equal", and there's a fine line between that and "greater than" (which you requested!), as bakunin already pointed out. We're still not clear WHAT you really mean.

8

Hi Rudic , zaxxon
Below is another example of what I am looking for :
Can you all please help ?

sample file.txt

Line,count,symbol
1,3,A
2,4,B
3,5,C
4,1,D
5,2,E
6,6,F
7,7,G
8,8,H

Expected result:
Group records based on the cumulative sum of second column "count" ,
such that the records fall under the same group till the cumulative sum of the coulmn "count" exceeds 12.
Once the cumulative sum exceeds , the loop should restart . for eg below , the first three records fall under group '1'
as their cumulative sum is 12.Similarily the records from 4 to 6 falls under group '2' as the cumulative sum (1+2+6)=9 which is less than 12.Similarily , the records 7 and 8 will be in groups 3 and 4 respectively

sample output

Line,count,symbol,group
1,3,A,1
2,4,B,1
3,5,C,1
4,1,D,2
5,2,E,2
6,6,F,2
7,7,G,3
8,8,H,4
9

Hello paul1234,

Could you please try following and let me know if this helps you.

awk -v val=1 -F, 'NR==1{print;next} {SUM+=$2} SUM>12{val++;SUM=$2} SUM<=12{print $0 FS val}'  Input_file

Thanks,
R. Singh

10

@bakunin,
you promote bash,
where the last part of a pipe is run in a subshell, so the following does not work

    echo "$line" | IFS=, read junk var1 junk
    echo "$var1"

It only works in ksh and its derivatives.
You can use a here string

    IFS=, read junk var1 junk <<< "$line"
    echo "$var1"

If there is a while loop anyway, consider to immediately apply the field splitting, because composing the line is easy

while IFS=, read field1 var1 last
do
    line="$field1,$var1,$last"
    echo "$line"
    echo "$var1"
done < file_size_num

.txt

11

Hi Ravinder ,
Its not working :frowning: grouping column is incorrect . PFB the output

-bash-4.2$ awk -v val=1 -F, 'NR==1{print;next} {SUM+=$2} SUM>12{val++;SUM=$2} SUM<=12{print $0 FS val}'  file_size_num1.txt
1,3,/informatica/apps/logs/fin/ote/TgtFiles/Destination/CWXX/CW1900
2,4,/informatica/apps/logs/fin/ote/TgtFiles/Destination/CWXX/CW1901,1
3,5,/informatica/apps/logs/fin/ote/TgtFiles/Destination/CWXX/CW1901,1
4,1,/informatica/apps/logs/fin/ote/TgtFiles/Destination/CWXX/CW1901,1
5,2,/informatica/apps/logs/fin/ote/TgtFiles/Destination/CWXX/CW1900,1
6,6,/informatica/apps/logs/fin/ote/TgtFiles/Destination/CWXX/CW1901,2
7,7,/informatica/apps/logs/fin/ote/TgtFiles/Destination/CWXX/CW1901,3
8,8,/informatica/apps/logs/fin/ote/TgtFiles/Destination/CWXX/CW1901,4
12

Hello paul1234, That's because your previous post's sample Input_file has headers and the data you are trying is NOT having it, so when you will have headers it should fly then, let me know how it goes then. Thanks, R. Singh

13

Hi Ravinder ,
Thank you so much .. I slightly changed the code that you provided and it seems to be working now.
Thank you all:)

 awk -v val=1 -F, 'NR==N-1{print;next} {SUM+=$2} SUM>12{val++;SUM=$2} SUM<=12{print $0 FS val}' < file_size_num1.txt
  
 output as expected:
1,3,/informatica/apps/logs/fin/ote/TgtFiles/Destination/CWXX/CW1900,1
2,4,/informatica/apps/logs/fin/ote/TgtFiles/Destination/CWXX/CW1901,1
3,5,/informatica/apps/logs/fin/ote/TgtFiles/Destination/CWXX/CW1901,1
4,1,/informatica/apps/logs/fin/ote/TgtFiles/Destination/CWXX/CW1901,2
5,2,/informatica/apps/logs/fin/ote/TgtFiles/Destination/CWXX/CW1900,2
6,6,/informatica/apps/logs/fin/ote/TgtFiles/Destination/CWXX/CW1901,2
7,7,/informatica/apps/logs/fin/ote/TgtFiles/Destination/CWXX/CW1901,3
8,8,/informatica/apps/logs/fin/ote/TgtFiles/Destination/CWXX/CW1901,4
14

Hello paul1234, You could THANKS button to thank any helpful post to any person here forum. I wanted to let you know that you need not to put NR==1 {print;next} since you don't have headings, if you have headers then you could add this condition to code. Thanks, R. Singh

16

hm, i thought that the lastpipe parameter to the shopt builtin did away with that, no? To quote the bash reference manual:

bakunin

17

The lastpipe option is available since bash 4.2
The man page looks promising.
But my practical experiment failed: the option had no such effect.

18

Watch it!:

While not working for me in an interactive shell, it does when encountered in a script (a subshell)...

1 Like
19

That's it. It works in a bash script - not on the command line!
The background jobs work nevertheless:

#!/bin/bash
shopt -s lastpipe
echo "1|2|3" | IFS="|" read a b c
echo "read result: $a $b $c"
sleep 10 &
jobs
wait
echo fin

Of course the interactive job controls like ^Z,bg,fg would not make sense.