Hi All,
I am trying to obtain count of characters using awk, but "length" function returns a value of 1 for 2-byte or 3-byte characters as well unlike wc -c command.
I have tried to use the below commands within awk function, but it does not seem to work
{
cmd="wc -c "stringtocheck
( cmd ) | getline Count
print Count
}
Can you please give me a solution. I have tried other options such as echo -e, print, but none seems to work.
Thanks & Regards
What exactly is going wrong? Do you want byte counts or char counts? As to awk , my version reports the same byte count as does wc -c :
wc -c <file
57
awk '{print length}' file
43
12
(You'll have to count the two <NL> chars in)
As to your awk function - wc needs an input file, so it won't count the stringtocheck variable...
Hi,
I need the byte count in a line (ignoring the newline character). Currently wc -c is giving me the byte count and length function within awk is giving me the character count.
Thanks & Regards
Hi tostay2003...
If awk is so important then why not combine the two?
Longhand using OSX 10.7.5, default bash terminal:-
Last login: Tue Dec 1 13:06:10 on ttys000
AMIGA:barrywalker~> cd Desktop/Code/Shell
AMIGA:barrywalker~/Desktop/Code/Shell> ls -l binary
-rw-r--r-- 1 barrywalker staff 8000 28 Jan 2015 binary
AMIGA:barrywalker~/Desktop/Code/Shell> hexdump -C binary
00000000 4c 19 5e a6 0a 10 2e 20 9a 3e f8 be e8 6c e4 bd |L.^.... .>...l..|
00000010 a7 f2 0c ca 66 dd 24 23 ba 01 9d 98 24 99 69 89 |....f.$#....$.i.|
00000020 30 73 e0 06 da 35 b8 91 8b cc ea 81 6a b6 ff c3 |0s...5......j...|
00000030 5d b2 67 2a 53 dc 53 33 79 f2 15 ad de e8 e0 6a |].g*S.S3y......j|
00000040 fc 5e 64 a2 8f 50 e8 10 b3 dd 35 23 e3 3d 5b a2 |.^d..P....5#.=[.|
.
.
.
00001f00 52 63 7a 06 34 f3 04 b4 a9 77 6f e9 6b 5b 1b 1d |Rcz.4....wo.k[..|
00001f10 dd 08 6e 65 47 9f a0 5f e9 af cb 37 95 d6 02 c0 |..neG.._...7....|
00001f20 c3 10 53 20 33 94 7f b5 04 47 1c ac c4 b7 b0 06 |..S 3....G......|
00001f30 97 4f 84 be 95 53 1e d8 09 15 b8 20 f2 8d 42 cf |.O...S..... ..B.|
00001f40
AMIGA:barrywalker~/Desktop/Code/Shell> awk 'BEGIN { system ( "wc -c < binary" ) }'
8000
AMIGA:barrywalker~/Desktop/Code/Shell> _
EDIT:-
Apologies...
Ignore the above as I see that you want binary byte count per line not per file.
My error...
Is this what you are after?
#!/bin/bash
# blc
echo "���������asdfh����
����������������AJHKK����������
�����������������Ԉ�1234567�����������" > /tmp/text
cat /tmp/text
hexdump -C /tmp/text
linenumber=1
while read -s -r line
do
length=$( echo "$line" | wc -c )
echo "Line $linenumber length less newline = $(($length-1))..."
linenumber=$((linenumber+1))
done < /tmp/text
Resilts:-
Last login: Tue Dec 1 13:07:20 on ttys000
AMIGA:barrywalker~> cd Desktop/Code/Shell
AMIGA:barrywalker~/Desktop/Code/Shell> ./blc
���������asdfh����
����������������AJHKK����������
�����������������Ԉ�1234567�����������
00000000 e2 80 a2 c2 b6 c2 a7 e2 80 a2 c2 b6 e2 80 a2 c2 |................|
00000010 b6 c2 a7 e2 80 a2 61 73 64 66 68 e2 88 86 cb 99 |......asdfh.....|
00000020 e2 88 86 cb 99 c2 a9 c2 a9 cb 99 e2 88 86 cb 99 |................|
00000030 e2 88 86 cb 99 c2 a9 e2 88 86 cb 99 c3 a6 0a e2 |................|
00000040 80 a0 c2 ae e2 80 a0 c2 a5 c2 a5 e2 80 a0 c3 b8 |................|
00000050 cb 99 cb 9a e2 88 86 cb 99 e2 88 86 cb 99 c2 a9 |................|
00000060 cb 9a e2 88 86 c3 b8 e2 80 a2 c2 b6 c2 a7 c2 b6 |................|
00000070 c2 a7 c2 a2 c2 a7 e2 88 9e 41 4a 48 4b 4b c2 a7 |.........AJHKK..|
00000080 e2 80 a2 e2 80 a2 c2 ba c2 ba c2 a9 c6 92 c6 92 |................|
00000090 c2 a9 e2 88 9a e2 88 ab c3 a7 e2 88 ab 0a ef a3 |................|
000000a0 bf c3 94 ef a3 bf c3 94 c3 93 ef a3 bf c3 94 ef |................|
000000b0 a3 bf c3 82 cb 9c ef a3 bf c2 a8 c3 81 cb 86 c2 |................|
000000c0 a8 c2 b0 ef ac 82 cb 87 c3 93 cb 9d cb 9d e2 97 |................|
000000d0 8a c3 94 c3 93 ef a3 bf c4 b1 ef a3 bf c3 94 ef |................|
000000e0 a3 bf c3 94 c3 93 c3 94 c3 93 c3 94 cb 86 c2 a8 |................|
000000f0 31 32 33 34 35 36 37 cb 86 cb 86 c3 81 cb 86 c2 |1234567.........|
00000100 a8 c3 81 cb 86 c2 a8 cb 86 c2 a8 c3 81 0a |..............|
0000010e
Line 1 length less newline = 62...
Line 2 length less newline = 94...
Line 3 length less newline = 111...
AMIGA:barrywalker~/Desktop/Code/Shell> _
The description of your problem is extremely confusing. UTF-8, UTF-16, and UTF-32 are completely different character sets and if you have a single file that contains characters from all three, determining which bytes in that file represent a <newline> character may be impossible unless you can clearly describe byte offsets in your file where there are shifts from one codeset to another and clearly describe how any program reading this file can determine what codeset is in use for any particular byte in that file.
If you are reading a file that is entirely encoded in UTF-8 (in which characters can be encoded with one to six bytes), you could tell your script that the UTF-8 input file was instead a file encoded in ISO 8859-1 (in which all characters are one byte) and count characters in lines in awk using the length() function since the <newline> character is encoded the same way in both codesets.
But, since you haven't described what the rest of your awk program is doing, we have no way to guess at whether or not this option might work for you and no way to guess if there might be other options.
Although Don has very valid points this might be a starting point...
Again OSX 10.7.5, default bash terminal.
1) Manually generate a UTF8(ish) test file named /tmp/text.
echo "���������asdfh����
����������������AJHKK����������
�����������������Ԉ�1234567�����������" > /tmp/text
2) Create a working stand alone shell script.
#!/bin/bash
# blc.sh '/full/path/to/filename'
linenumber=1
while read -s -r line
do
length=$( echo "$line" | wc -c )
echo "Line $linenumber length less newline = $(($length-1))..."
linenumber=$((linenumber+1))
done < $1
3) Fit this inside an awk script.
#!/bin/sh
# blc_awk.sh /full/path/to/filename
# Note there must be a whitespace after blc.sh.
awk -v arg="$1" 'BEGIN { system ( "~/Desktop/Code/Shell/blc.sh "arg ) }'
Run in this case as blc_awk.sh /tmp/text
(In this example my blc.sh sits in the directory shown in the script just change yours to suit.)
Hope this is useful...
Results:-
Last login: Tue Dec 1 21:08:13 on ttys000
AMIGA:barrywalker~> cd Desktop/Code/Shell
AMIGA:barrywalker~/Desktop/Code/Shell> echo "���������asdfh����
> ����������������AJHKK����������
> �����������������Ԉ�1234567�����������" > /tmp/text
AMIGA:barrywalker~/Desktop/Code/Shell> ./blc.sh /tmp/text
Line 1 length less newline = 62...
Line 2 length less newline = 94...
Line 3 length less newline = 111...
AMIGA:barrywalker~/Desktop/Code/Shell> ./blc_awk.sh /tmp/text
Line 1 length less newline = 62...
Line 2 length less newline = 94...
Line 3 length less newline = 111...
AMIGA:barrywalker~/Desktop/Code/Shell> _
I read binary with awk a lot... LANG=C (environment var) is your friend.