Replace Control M (^M) character with new line

Hi All,
We are getting an external file in abc.csv format. When opened in excel spread sheet, it is opening alright. But when opened in notepad, I see the contents in a single line.
Ftp'd the file in binary mode to AIX UNIX host. When opened, I see Control M (^M) characters in place of New Line characters.

I did the following:
$ wc -l abc.txt
0 abc.txt
$ sed 's/^M//g' abc.txt > xyz.txt
$ sed 's/^M$//' abc.txt > xyz.txt

I'm unable to replace the ^M character with new line. Can you kindly help me with the solution?

Thanks in advance.

dos2unix abc.txt abc.txt

will remove control ^M characters

Makarand, we don't have this utility. Any other way to replace he ^M characters in shell script?

if you have "Editplus" then save as the file and try choose file format as "Unix"
if you have "or "Notepad++" then save as the file as "unix script file"
OR
open the file in wordpad and instead of FTP it ; try to copy the file contents and then vi filename & paste in Unix

Sorry Makarand but I need to do this as part of the shell script.

I also tried the below.

sed 's/'"$(printf '\015')"'//g' abc.txt > xyz.txt

It removed the ^M character but it is not replaced with newline character.

Please note that the original file is showing all data in single line with new line being displayed as ^M. When I do "wc -l abc.txt", it displays 0 rows indicating it is not recognising any new line characters.

My goal is to introduce these newline characters so that I can process the data row by row

Thanks!!

What is the output of this command:

file abc.txt

abc.txt: ascii text

A search in this forum can help you.

http://www.unix.com/shell-programming-and-scripting/28780-trying-remove-m-characters-file-2.html

In addition to the other advice that you received, you should also search for "perl one liners remove control M". There seems to be no end to the number of things that can get done with perl one liners.

I'm assuming that abc.csv was created from excel on a mac in which case this should work:

sed 's/'"$(printf '\015')"'/'"$(printf '\012')"'/g' abc.txt > xyz.txt

Try looking for the dos2unix command in /usr/bin/dos2unix.
It's possible that you don't have /usr/bin in your path.

You can also try echo $PATH

Thanks to all that tried to help me.

I didn't find /usr/bin/dos2unix on our UNIX host.

xbin - the cmd didn't work.

I tried the below cmd...seems working..

$ sed 's/'"$(printf '\015')"'/\
> /g' abc.csv > xyz.csv

Note that after first "\" (back slash), I pressed return key (Enter key) that took me to the next line with ">" symbol. This seems to have replaced ^M character with newline. I'll confirm/reply after my testing.

According to forum rules, you should search the forum for asking questions. Obviously, this question has been asked and answered a number of times, including:

Which one is faster to remove control m characters?

Yes it worked!

In the shell script, when I pressed Enter key, it didn't display ">" symbol but started the next characters from new line....

$cat Test.ksh
#!/bin/ksh

sed 's/'"$(printf '\015')"'/\
/g' abc.csv > xyz.csv

---------- Post updated at 02:40 PM ---------- Previous update was at 02:38 PM ----------

Neo - I did searched the forum before posting my query. Pls see what commands I tried in the initial post under this thread. I got those leads from searching the form itself.

I posted my query only when the answers on the other questions didn't help me.

Hope you understand.

Thanks!

---------- Post updated at 02:45 PM ---------- Previous update was at 02:40 PM ----------

How can I mark this thread as solved?

See attached image on how to mark threads as solved.

The strings command will do the trick,

strings binary_file

I would suggest re-sending the FTP as an ASCII file and remove the problem altogether.

Does that help and save you the unnecessary follow up work?

Robin