Getting an unexpected newline in my while loop line-by-line feed

Shell Programming and Scripting
1

Hi,
I'm trying to get a line returned as is from the below input.csv file in Bash in Linux, and somehow I get an unexpected newline in the middle of my input.

Here's a sample line in input.csv

$> more input.csv
TEST_SYSTEM,DUMMY@GMAIL.COM|JULIA H|BROWN

And here's a very basic while loop to read the contents of input.csv:

INPUT=`cat input.csv`
for read in $INPUT; do
	echo "current line: " $read
done

Here is the output:

current line:  TEST_SYSTEM,DUMMY@GMAIL.COM|JULIA
current line:  H|BROWN

I noticed that when I remove ' H' from the input (if the input.csv line reads

TEST_SYSTEM,DUMMY@GMAIL.COM|JULIA|BROWN),

then the whole line gets printed fine. So there's something fishy with the ' H' format that's driving me nuts.

Thanks.

2

It's easy to see from your sample input that the issue is the space between the A and the H . You have not written your script in any way that it knows the input file is a CSV file, so your post is a bit confusing.

What did you expect the output to be, exactly, based on the script you posted?

3

Some more comments:

  • it's not wise (though not an error) to use read for a variable name as it is a bash (builtin) command.
  • it might be worthwhile to read about the IFS variable in bash .
  • why the detour over cat and a variable? Why a for loop at all?
  • .csv stands for "comma separated values" - pipes as separators are possible, but quite unusual.
4

In addition to what RudiC so succinctly analysed: you fell victim to something that is called "field splitting" in the shell. I suggest you look up the concept, it will make the meaning of the suggested IFS variable much clearer.

I hope this helps.

bakunin

5

Thanks guys, I will look into 'field splitting' concept and figure out the IFS command.

--- Post updated at 10:49 AM ---

Here is what works for me:

printf '%s\n' "$INPUT" |
  while IFS= read -r line; do printf '%s\n' "$line"; done

Thanks again.

6

Easier and without detours:

while IFS= read line ; do
     printf "%s\n" "$line"     # change this to the actual processing code
done < /path/to/your/file

For "field splitting" you might want to start here: Help with file compare and move script. See post #6 in this thread

I hope this helps.

bakunin

7 
INPUT=`cat input.csv`
for read in $INPUT; do
	echo "current line: " $read
done

need to quote variable:

INPUT=`cat input.csv`
for read in "$INPUT"; do
	echo "current line: " $read
done

if more than one line

mapfile -t < <(cat input.csv)
for read in "${MAPFILE[@]}"; do
        echo "current line: " $read
done
8

Why do any of those when the while read does the exact same without the cat and without any bash-only features?