Convert delimited to fixed length

Shell Programming and Scripting
1

Hi, I have to change a tab delimited file to a fixed length file. For text fields I need to left justify and NULL fill to the right and for number fields I need to right justify and zero fill to the left. If there are spaces between words in a text field I need to keep them as spaces. I am using korn shell and AIX. Here is a portion of a flat file that I am working with (First field is text and 12 characters long and second is a number 10 characters long) -
Bob Smith<<tab>>139.90
Kathy Reys<<tab>>-40.50

Here is the output that I need -
Bob Smith 0000139.90
Kathy Reys 0000-40.50

2

What an odd way to handle negative numbers...

#! /usr/bin/ksh
typeset -L13 name
typeset -R10 val2
IFS="<<tab>>"
exec < datafile
while read name val ; do
         val2="0000000000000000"$val
         echo "$name $val2"
done
exit 0
3

Thanks for that help, that got me most of the way to where I needed. Do you know how to do the NULL padding after the strings instead of space padding? I thought I figured it out with a -Z on the typeset, but it doesn't seem to be working. Thanks again for the help.

4

Sheesh, what a crazy format. ksh uses strings which are null terminated and this makes working with nulls virtually impossible. We can pad with a substitute character and then translate all occurences of the sunstitute character to nulls as the last operation.

#! /usr/bin/ksh
typeset -L13 name2
typeset -R10 val2
PAD="%"
NULLS="${PAD}${PAD}${PAD}${PAD}"
NULLS="${NULLS}${NULLS}${NULLS}${NULLS}"
IFS="<<tab>>"
exec < data
while read name val ; do
        val2="00000000000000"$val
        name2=${name}${NULLS}
        echo "$name2 $val2"
done | tr $PAD "\000"
exit 0
5

Thanks for your help.

6 
ruby -F\011 -nale'print $F[0].ljust(12,0.chr),$F[1].rjust(10,"0")' file
7

I thought I had this figured out, but I'm running into another scenario that I need help with. Do you know how I can fix this if I have no data in a field, <<tab>><<tab>>? What is happening is that it's just going to the next field where there is data and using that, but I need it to use the empty field. Not sure if I explained that well, so here's an example -
Input file - (first field left justified NULL padded (10bytes), second field left justified NULL padded (1 byte), and third field left justified NULL padded (10bytes))
Bob Smith<<tab>><<tab>>Susan Smit
Kathy Reys<<tab>>A<<tab>>Ron Davis

Output file currently getting -
Bob Smith<<NULL>>Susan Smit
Kathy ReysARon Davis<<NULL>>

Needed output -
Bob Smith<<NULL>><<NULL>>Susan Smit
Kathy ReysARonDavis<<NULL>>

Hopefully that makes sense, if not I can try to explain more.
Thanks.

8 
ruby -F\t -nale'print $F[0].ljust(10,0.chr),$F[1].ljust(1,0.chr),
$F[2].ljust(10,0.chr)' file
9

And for those without ruby...

#! /usr/bin/ksh
typeset -L13 name
typeset -R10 val
PAD="%"
NULLS="${PAD}${PAD}${PAD}${PAD}"
NULLS="${NULLS}${NULLS}${NULLS}${NULLS}"
IFS="   "
exec < data
while read  line; do
        set - $line
        val="00000000000000"$2
        name=${1}${NULLS}
        echo "$name $val"
done #  | tr $PAD "\000"
exit 0
10

I still seem to be getting the same results as before. I have dug through every book I have and I cant figure out how to fix this. It's still skipping over empty fields instead of setting them to NULLs.

11

Post a small sample of input data that fails. I cannot get my latest script to fail.

12

Here is a very small sample file with 3 fields. The first record has data in field 1 and field 3 and the 2nd field is empty and the second record has data in field 1 and 2 and field 3 is empty.

13

Neither record has any numeric data in it. I do not understand how this data relates to the original problem which was:
Bob Smith<<tab>>139.90
Kathy Reys<<tab>>-40.50
Two fields per record, not three.
Second field was numeric.

14

I'm sorry, I should have explain it better. As I was trying to work with more data and more files I ran into this new scenario. I maybe should have opened a new thread, but I wasn't sure. This is a new data scenario, but using the same solution that you originally sent me, adding another field, and changing the second and third field to text.

15

Try this technique

#! /usr/bin/ksh
typeset -L13 name2
typeset -R10 val2
PAD="%"
NULLS="${PAD}${PAD}${PAD}${PAD}"
NULLS="${NULLS}${NULLS}${NULLS}${NULLS}"
IFS="$PAD"
exec < data
sed 's/<<tab>>/'"$PAD"'/' | while read  name val; do
        val2="00000000000000"$val
        name2=${name}${NULLS}
        echo "$name2 $val2"
done | tr $PAD "\000"
exit 0