How to convert a file containing hex code to decimal using script?

Shell Programming and Scripting
1

The file contains code like the below and need to convert each one into a decimal

00 00 00 04 17 03 06 01

So the output should come as

0 0 0 4 23 3 6 1

2 
$ for i in 00 00 00 04 17 03 06 01 ; do echo "ibase=16; $i" | bc ; done
3

Without external programs

for i in 00 00 00 04 17 03 06 01 ; do
  printf "%d\n" "0x$i"
done
4

Hi Jayan

I have a file called hex.txt
the file contains numerous entries i, just gave 8 as an example , I need to pick up entire content from the file and convert to decimal and not just those 8 alone

Can u advise if that is possible

5

Please post a representative sample of hex.txt and the desired output..

6

hex.txt contains numerous entries like this

00 00 00 04 0A 02 05 01 00 00 00 00 0A 0A 32 C7 00 00 00 00 0A 0A 32 F3 0A 0A 2B 2D 00 00 00 00 00 00 00 08 80 01 06 6D 26 E4 00 02 00 00 00 00
06 6D 0A 80 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

So for eg : 00 00 00 04 0A 02 05 01 should be converted into decimal form 0 0 0 4 10 2 5 1 and so on

7

Try:

awk '{for(i=1;i<=NF;i++)$i=sprintf("%0d","0x"$i)}1' infile
8

The conversion didnt occur

-bash-3.2$ awk '{for(i=1;i<=NF;i++)$i=sprintf("%0d","0x"$i)}1' Hextxt
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
-bash-3.2$
9

Try with this ..

$ for i in $(cat infile) ; do echo "ibase=16; $i" | bc ; done | xargs
1 Like
10

What do you get when you run it on your own sample?
I get:

$ awk '{for(i=1;i<=NF;i++)$i=sprintf("%0d","0x"$i)}1' infile
0 0 0 4 10 2 5 1 0 0 0 0 10 10 50 199 0 0 0 0 10 10 50 243 10 10 43 45 0 0 0 0 0 0 0 8 128 1 6 109 38 228 0 2 0 0 0 0
6 109 10 128 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

What OS and version are you using?

11

Thanks Jayan , that worked

Just need to format it now .I will get back on this

Scrutinizer , havent checked the OS yet

12

If using gawk, then try option --non-decimal-data

$ awk '{for(i=1;i<=NF;i++)$i=sprintf("%0d","0x"$i)}1' infile
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

$ awk --non-decimal-data '{for(i=1;i<=NF;i++)$i=sprintf("%0d","0x"$i)}1' infile
0 0 0 4 10 2 5 1 0 0 0 0 10 10 50 199 0 0 0 0 10 10 50 243 10 10 43 45 0 0 0 0 0 0 0 8 128 1 6 109 38 228 0 2 0 0 0 0
6 109 10 128 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

$ awk --version
GNU Awk 3.1.7
1 Like
13

Thanks Ygor, that solves the mystery. Another gawk quirk, it seems..

awk --posix

also seems to do the trick, perhaps for different reasons? It looks like gawk (as an extension) is interpreting these constants, while reading them, whereas I want them to be treated as simple strings and let printf do the conversion...

14

I have written the data into hexdata.log output as for one of the entries

0 0 0 4 235 20 192 65 0 0 0 0 10 20 40 99 10 10 34 24 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 152 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

Need to be able to pull out words , these are actually IPs
5-8 -235.20.192.65
13-16 -10.20.40.99
16-20 - 10.10.34.24
29-33 - 0 ( C -Count )

So the output file reads as

245.20.192.65 10.20.40.99 10.10.34.24 c:0
15

Try...

awk 'function p(i){printf "%d.%d.%d.%d ",$i,$(i+1),$(i+2),$(i+3)}{p(5);p(13);p(17);print "c:" $29+$30+$31+$32+$33}' hexdata.log
1 Like
16

Thanks very much Ygor , i am getting desired output

235.20.192.65 10.10.50.9 10.10.39.42 c:0

but this steps need to repeat for each and every line in the file (each line as same number of words 108)
Right now it only checks the first line and gives the output

---------- Post updated 06-01-12 at 12:16 AM ---------- Previous update was 05-31-12 at 11:56 PM ----------

Also need the time stamp to be printed at the beginning of the line

---------- Post updated at 01:49 AM ---------- Previous update was at 12:16 AM ----------

guys , Please can you help out with the above ,

I really appreciate the efforts taken by all your efforts so far

17

Not sure what you mean by a time stamp?

Anyway, combining the awk code above gives you something to try...

$ cat hex.txt
00 00 00 04 0A 02 05 01 00 00 00 00 0A 0A 32 C7 00 00 00 00 0A 0A 32 F3 0A 0A 2B 2D 00 00 00 00 00 00 00 08 80 01 06 6D 26 E4 00 02 00 00 00 00
06 6D 0A 80 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

$ cat prog.awk
        function p(i)
        {
                printf "%d.%d.%d.%d ", $(i), $(i + 1), $(i + 2), $(i + 3)
        }

        {
                for (i = 1; i <= NF; i++) {
                        $i = sprintf("%0d", "0x" $i)
                }
                printf "%s ", ts
                p(5)
                p(13)
                p(17)
                print "c:" ($29 + $30 + $31 + $32 + $33)
        }

$ gawk --posix -f prog.awk ts=$(date '+%Y%m%d%H%M%S') hex.txt > result.txt

$ cat result.txt
20120601110632 10.2.5.1 10.10.50.199 0.0.0.0 c:0
20120601110632 0.0.0.0 0.0.0.0 0.0.0.0 c:0

$
18

Thanks ygor , by timestamp i meant the current time , i tried the steps you mentioned but it gives all zeroes

-bash-3.2$ gawk --posix -f prog.awk ts=$(date '+%Y%m%d%H%M%S') Hex.txt
20120602173822 0.0.0.0 0.0.0.0 0.0.0.0 c:0
20120602173822 0.0.0.0 0.0.0.0 0.0.0.0 c:0
20120602173822 0.0.0.0 0.0.0.0 0.0.0.0 c:0
19

try option --non-decimal-data

20

Ygor ,How do i user that option ?where do i set it