Getting rid of ^M

Shell Programming and Scripting
1

I have a text file with hundreds of 32-character hash codes in it, each terminated with a linefeed (/l, or ^M).

185ead08e45a5cbb51e9f7b0b384aaa2
57643e1a17252a9fc746d49c3da04168
60cba11d09221d52aaabb5db30f408a2
2b75ee6e5c2efc31b4ee9a190d09a4df

...... etc.

I want to create a file for each hash code, with the .txt extension.

I ran a test shell script called test1.sh to see if I could pull out each line in the file and create a file name from it:

#!/bin/sh
   while read line
   do
      echo $line.txt
  done < hashlist.txt
echo "done"

What I got was

$ ./test1.sh | less

  185ead08e45a5cbb51e9f7b0b384aaa2^M.txt
57643e1a17252a9fc746d49c3da04168^M.txt
60cba11d09221d52aaabb5db30f408a2^M.txt
2b75ee6e5c2efc31b4ee9a190d09a4df^M.txt
...... etc.

How can I create these file names without the ^M?
It seems if I remove the ^M and replace it with a carriage return (/r) then the while loop won't work.

Teledon

2

Convert the file first using dos2unix or dos2ux utility, if you can't find it (would be surprised though...) I will give you some alternatives (scripts?).
After conversion you should not have any trouble...

P.S.
Next time, use code tags for your code and data

3

You can use " perl -pi -e 's/\r//g' file_name " (dont use double quotes) to get rid of the control m characters.

4

If you imported your file from MS win... world , you have the reasom of the presence of these char on end of line...(thoufh some clever ftp utilities do the conversion for you...)

5

Or remove the extra characters with "tr" as you read the file.

#!/bin/sh
cat hashlist.txt | tr -d '\r' | while read line
do
      echo "${line}.txt"
done
echo "done"

As I'm sure you realise, the spurious carriage-return is Microsoft Text File format. How you transfer a file to a unix platform is important (e.g. use text mode ftp not binary mode ftp).

1 Like
6

tr is nice cpu friendly little tool to convert chars or delete chars.

#!/bin/sh
# \015 = CR
cat hashline.txt | tr -d '\015' | while read line
do
      echo $line.txt
done
echo "done"
7

[LEFT]To remove the ^M characters at the end of all lines in vi, use the following command:

[COLOR=\#0000ff]

shift :%s/^V^M//g
1 Like