Replacing Hex Characters In A File Using awk?

UNIX HP-UX
1

Hi guys,

First off, i'm a complete noob to UNIX and LINUX so apologies if I don't understand the basics!

I have a file which contains a hex value of '0D' at the end of each line when I look at it in a hex viewer.

I need to change it so it contains a hex value of '0D0A0A'

I thought the easiest way would be using awk so I tried this

awk '{sub("\x0D","\x0D\x0A\x0A")}' filename

It doesn't give me an error but neither does it seem to work. Can anyone tell me what i'm doing wrong?

Thanks for any advice!

2

Could you post a sample of the file?

3

try

cat filename| sed 's/0D/0D0A0A/g' > file2 && mv file2 filename
4

Here is an example of the file in question

 
^A{1:F01BNPAGB20AXXX0000000000}{2:I103BARCGB20XXXXN}{3:{108:ORLD190077833XXX}}{4:^M
:20:CCT44195^M
:23B:CRED^M
:32A:140428GBP627,95^M
:33B:GBP632,95^M
:50K:/CH5708686001085122004/TR32793333^M
MSC CROCIERE SA^M
CHEMIN RIEU 12-14^M
GENEVE//CH^M
:52A:BPPBCHGGXXX^M
:57A:BARCGB20XXX^M
:59:/GB65BARC20789880480169^M
Gorkana Group Ltd^M
Discovery House,  28-42 EC1Y8QE Lon^M
don//GB^M
:70:MSC CROCIERE SA - Gorkana Group Ltd^M
. RIF. 647413 30/11/2013^M
:71A:SHA^M
:71F:GBP5,^M
-}^C                                                         ^A{1:F01BNPAGB20AXXX0000000000}{2:I103BARCGB20XXXXN}{3:{108:ORLD190077835XXX}}{4:^M
:20:CCT44196^M
:23B:CRED^M
:32A:140428GBP2703,88^M
:33B:GBP2708,88^M
:50K:/CH5708686001085122004/TR32793355^M
MSC CROCIERE SA^M
CHEMIN RIEU 12-14^M
GENEVE//CH^M
:52A:BPPBCHGGXXX^M
:57A:NWBKGB2LXXX^M
:59:/GB94NWBK50410145129827^M
Worldchoice Enterprises Ltd^M
Worldchoice House PE2 6FT PETERBORO^M
UGH//GB^M
:70:MSC CROCIERE SA - Worldchoice Enter^M
prises Ltd. RIF. 8375 21/10/2013^M
:71A:SHA^M
:71F:GBP5,^M
-}^C                                     ^A
5

Try:

tr -d '\001' < file > newfile

To remove it. In the first post you want to change it to something, but I am not sure to what..

Could you run a relevant part go your file through od -c ?

6

I am trying to change a hex value of 0D to 0D0D0A so I really need to replace one character with three characters which is what I think is making it tricky.

7

It seems to me that there lines in DOS format, that are ending with CR (0D) - LF (0A) , and there are also FF characters (0C). But what do you want to do with them? Why do you want to change 0D to 0D0D0A ?

8

OK, it appears I have muddied the waters by only explaining what I think I need.

Given that my knowledge is poor at best i'll explain the whole problem to try and give it context which will hopefully help.

I have a file which contains a hex value of 01 at the beginning of each 'record' and a hex value of '03' at the end of each record.

Within each record there are several 'lines' of data that end with a hex value of '0D0A'

So the data would look like this (crlf signifies a carriage return/line feed)

^AHIcrlf
BYEcrlf
-}^C

If I look at it in a HEX viewer I see

01 48 49 0D 0A
42 59 45 0D 0A
2D 7D 03

The problem I have is that the application I need to put the file into has the following restrictions

  • The ^A character(hex value of 01) needs to be replaced with a space(hex value 20). So, I want to replace hex 01 with hex 20

  • The ^C character(hex value 03) needs to be replaced with a carriage return and two line feeds(hex values 0D 0A 0A). So, I want to replace hex 03 with hex 0D 0A 0A.

What I want to end up with is

20 48 49 0D 0A
42 59 45 0D 0A
2D 7D 0D 0A 0A

Hopefully that makes what i'm aiming for a little clearer and apologies for not being accurate enough initially.

9

Looks like your file is form DOS/win ... Try first using dos2ux then tell us what is left that needs modification...

10

Try:

awk '{gsub(/\001/,FS); gsub(/\003/,"\r\n\n")}1' file > newfile
11

That worked like a charm thank you!