Can ksh read records with blank fields

Shell Programming and Scripting
1

I have a tab delimited file with some fields potentially containing no data. In ksh 'read' though treats multiple tabs as a single delimiter. Is there any way to change that behavior so I could have blank data too? I.e. When encountering 2 tabs it would take it as a null field? Or do I have to use awk?

# where <TAB> would be a real tab:
while IFS="<TAB>" read a b c d; do echo $c; done < file.txt

vs.

awk -F"\t" '{print $3}' file.txt

The shell version will output the wrong field if the 1st or 2nd record is blank.

2

Yes try setting IFS to two consecutive hard tabs:

while IFS="<TAB><TAB>" read ...

and it is still best to use quotes around the variable expansion:

echo "$c"

--
Note: in ksh93 you could also use:

while IFS=$'\t\t' read
3

Scrutinizer,
I think the OP is saying the input file has lines with 4 fields using <tab> as a separator and some fields are empty. Unfortunately, when IFS is set to <tab> (or <space>) groups of <tab>s (or <space>s and <tab>s) are treated as a single field separator. I don't believe that this behavior for IFS set to a single <tab> character conforms to the standards, but since this seems to be existing practice in both bash and ksh , this may be considered a bug in the standard.

benalt,
If I understand what you're trying to do, the following should work with ksh (or any shell that performs the parameter expansions required by the POSIX standards):

#!/bin/ksh
while IFS='' read -r line
do	printf 'input:"%s"\n' "$line"
	d=${line##*	}
	line=${line%	*}
	c=${line##*	}
	line=${line%	*}
	b=${line##*	}
	line=${line%	*}
	a=${line##*	}
	printf 'a=%s,b=%s,c=%s,d=%s\n' "$a" "$b" "$c" "$d"
done < file.txt

(where the whitespace in % * and in * } is a single literal <tab> character in all of the above parameter expansions.

If file.txt contains:

a	b	all fields present	d
a		field 2 empty	d
	b	field 1 empty	d
		fields 1 & 2 empty	d
		fields 1, 2, & 4 empty	
a	b	c	d \
e	f	g	h

the output produced is:

input:"a	b	all fields present	d"
a=a,b=b,c=all fields present,d=d
input:"a		field 2 empty	d"
a=a,b=,c=field 2 empty,d=d
input:"	b	field 1 empty	d"
a=,b=b,c=field 1 empty,d=d
input:"		fields 1 & 2 empty	d"
a=,b=,c=fields 1 & 2 empty,d=d
input:"		fields 1, 2, & 4 empty	"
a=,b=,c=fields 1, 2, & 4 empty,d=
input:"a	b	c	d \"
a=a,b=b,c=c,d=d \
input:"e	f	g	h"
a=e,b=f,c=g,d=h
1 Like
4

Hi Don, you are right, I always thought this was part of the POSIX standard, but on rereading it seems to be specific to ksh93 and zsh . It does not work in ksh88 .

In man ksh93 :

$ ksh93 -c 'echo "  field3 field4  filed6" | { IFS="  " read a b c d e f g; echo "$d" ;}'
field4
$ zsh -c 'echo "  field3 field4  filed6" | { IFS="  " read a b c d e f g; echo "$d" ;}'
field4

So the answer to the original question is, yes with ksh93, no with ksh88.

1 Like