removing spurious charactors from file

Shell Programming and Scripting
1

Morning,

Hopefully someone can help an newbie!

I have been getting a number of file corruptions with
Rogue "Control M" charactors [ carriage return ].

The effect is:

Should be

aaaa
bbbb
cccc
dddd

(4 lines)

But getting

aaaa
bb<Ctl M>
bb
ccccc
dddd

(5 lines)

I am having to correct this manually using vi...

Anyone know a way to do this in a script?

Effectively I want to delete any occurrances of the string
"^M\n"

I can't use tr as

tr -d "^M" leaves two lines.
tr -d "^M\n" also deletes the genuine "\n" at end of line.

Any suggestions? sed?

The problem with manuals is that you need to know what to look up!

2

in vi:

esc. to cmd mode, then enter

:%s/CTRL+V CTRL+M//g

Note I left a space betw. the 2 CTRL sequences for readability, you shouldn't have any spaces.

3

Thanks.

However, I am able to do this using vi.

I am asking how to do it in a script,
so it can be automated.

4

Sorry I relied too much on your title and assumed it was a single file.

Try playing with this:

for fl in *.php; do
mv $fl $fl.old
sed 's/FINDSTRING/REPLACESTRING/g' $fl.old > $fl
#rm -f $fl.old
done

You could also do it something like:

for i in `find ./ -type f -exec grep -l "yourstringhere" {} \;`
        do 
        # use our sed rules to make changes. put them in a temp file
        sed -f ./sedlist $i > $i.tmp
        # this is optional. make a back up to be safe.
        #cp $i $i.sedbak
        # move the new temp file over the original
        mv $i.tmp $i
        done

without the "sedlist" file

Of course, change the values to suit your needs.

5

Try...

awk '/\r/{sub("\r","");printf $0;next};1' file1 > file2

6

the problem is that the
corruption inserts a <Ctl M> charactor and throws a line.

Hence

instead of bbbb

we get :

bb<CtlM>
bb

By simply substituting <Ctl M> with ''
I get

bb
bb

when I want

bbbb

In vi I have to use "J" to concatinate the two lines after subsituting <Ctl M> for ''

Hence I think I need to substitute the string
^M\n with ''

but I can't get this to work

7

Eureka!

Thanks Ygor.

That's done the trick!

(albeit with nawk rather than awk!)