[AWK] read lines with \x00 symbol

Shell Programming and Scripting
1

I want to read a large (~1-4Gb) txt file with fields separated by "," and line separator "\n". Unfortunately, file contains \x00 (zero ASCII) symbols
AWK treats them as end of line + it ignores reminder of the line after the \x00.

As a simple example:
echo "\0060\0061\000\0060\0063" | nawk '{print $0}'
Output: 01
I would like to get: 0103

Is there a way to ignore \x00 in (n)awk?

Thank you.
Dmitriy

2

On Linux and FreeBSD

# echo "\0060\0061\000\0060\0063" | nawk '{print $0}'
\0060\0061\000\0060\0063
3

At SuSe with gawk:
echo "\0060\0061\000\0060\0063" | gawk '{print $0}'
\0060\0061\000\0060\0063

but at Solaris 8 it behaves as you described. So maybe use some dummy tool before '[n|g]awk' like 'tr' or 'sed'?

4

On Solaris you should use /usr/xpg4/bin/awk

5

Thank you, Guys. Your comments help me a lot.

I should have specify my OS from the begining. It is SUN with 5.9 OS (very old but my department is happy with it :frowning: )
finally, I've decided to use perl to replace \x00, smthng like

perl -p -i -w -e 's/\x{00}//g' 20030417.txt

takes a while (~1min) but does what I want

6

Well it did not help:

echo "\0060\0061\000\0060\0063" | /usr/xpg4/bin/awk '{print $0}'
01

How come you know that there would be other version of awk?

Regards

7

I just search this forum. :rolleyes: