include NULLs in line length check

Shell Programming and Scripting
1

Hello,
I am checking the length of each line of a fixed length file and making sure all lines are 161 length. My problem is that some files contain null characters which gets stripped out of my echo. How do I have the NULLs included in my check? (and I cannot replace or sub the NULL values with anything either).
Thanks for the help....

#!/bin/ksh
RECORD_LEN=161
RECORD_COUNT=0
while read -r line
do
ACTUAL=`echo "${#line}"`
printf "\n$line\n"
echo $ACTUAL
if [ ${RECORD_LEN} -ne $ACTUAL ]; then
exit;
fi
done < file

2

try using awk

awk '{if(length($0)!=161){exit}else{print length($0)}}' filename
3

Thanks - but that code actually stops at the first NULL and truncates the line length and fails the length check.
Any other ideas

4

your original logic also do the same so i wrote exit once length is not equal to 161..

5

my point was that both your code and mine are not working. Let me try to explain my problem again. I have a fixed length file that I am reading every line and making sure every line is exactly 161 length. Problem is that some lines contain NULL characters (or low 'MAINFRAME' values). These are not space values.

For example, lets say that my file has one record and has a NULL right in the middle at position 80. My script is returning 160 as length and throwing out the NULL altogether. Your awk code is stopping at the NULL and returning the line length as 79. I need code to return all characters including the NULL values.
I was thinking instead of trying to get the actual line length I could go to the end of each line and retrieve the line position from there. I think that would be better but cannot find relevant code for that.
Thanks

6 
wc -c datafile | read value junk
if [[  $$( $value % 161 )) -ne 0 ]] ; then
      echo "data file does not have 161 lrecl record(s)"
else
      echo "ok"
fi

If you have to check every single line try something like this

dd if=datafile of=testfile cbs=161 block
cksum datafile | read chk1 junk
cksum testfile | read chk2 junk
if [[ "$chk1" != "$chk2" ]] ; then
   echo data format error in datafile"
fi
7

Please post a sample of how the input file looks like...

8

uploaded example of input. You can see NULL values in TextPad editor. Don't use in notepad and wordpad - they show up as spaces. In UNIX vi - the values are ignored.

9

just realized one of my lines I edited was long. re-attached

10

I decided to strip out the NULLs and replace with spaces. I could not get tr to work either. Maybe an issue with Solaris??
Below perl code worked though:

perl -pe 's/\000/ /g' file > newfile

Thanks for help. It would still be nice to know if my original question could be solved.

11

You can write a small C program to do this or use tr...

tr -d "\000" <file