Merge lines into one

Shell Programming and Scripting
1

Source data file from oracle, terminated by ",". 'Cause some of fields have \r\n, then those lines were splitted into multiple lines in the expoted data file. Just vi this file, and found ^M. How to concatenate these line into one if it has a ^M at then end.

thanks,

2

In these cases it really helps if you supply sample input data and an example of what you want to be output after the text processing.

3

"2007-09-09","1-71J-1","2004-03-20 12:00:59","0-1","2006-01-16 17:27:19","1-2RRYX-6","N","xuwedong",,"xwd",,,,,,"tv","Mr.","No Match Row Id","mail","111111111",,"+8610666666",,"rmrb","^M
Building B 1503","100102",,"Active","0-5220",,"CN","Working Time",,"11111111111111X","Passport",,"Master","payment",,,,"travel",,,,

4

Hello there,

Even we also have same issue and could not find any solution yet...how did you fix your problem?

Pl post the fix...

THank you,
Cheers,
H

5

Try:

tr -d '\r' < your_file

Regards

6

Hello Frenkline for quick update

But when I try tr -d '\r' it removed my ^M but still keeps my line as follows

Special Olympics Missouri,None,Muller+Company,MULLE001,4498,MS-80124-58-SOMO-ME-US-Special Olympics Basketball - St Louis,,Marc Hale,C1-MS-IS,,45305,2/18/2008,3/9/2008,N/A,MySpace ROS Leaderboard (728x90) ,cpm,1.45,"434,483",0,0,0%,0,"4739 Belleview
Kansas City, KS 64112
United States
",Net 30,Bill on Delivery - 3rd Party,jsydnor,N/A,Associated Profile,N/A,0,0,0,0,N/A,0.00%,N,February 2008 - February 2008,"248,276",360,0,0,N/A,0,0%,0,,N/A,N/A,N/A,"16 - 18, 18 - 24",N/A,8,N/A,United States,St.Louis-609,N/A,N/A,Feb-08,2/29/2008,2/1/2008,USD,2/18/2008,3/9/2008,,,,3832,0,452,100%,,,FIM,MySpace Standard Terms,0,,,N/A,N/A,N/A,N/A

After Belleview, Kensas City is on new line and United States also new line

I want that merge into one line.

I tried tr -d '\r' and '\n' also but still no help :frowning:

Thank you,
Cheer,
H

7

can you do something like:

head -2 filename | od -An -t oC -w10
or
head -2 filename | od -An -t dC -w10 <<--displays data in decimal ASCII

Thus, if I can do the following to see my data

> head -2 uprlwr.fmt                    
KEYCODE,25,C
FILEID,15,C

But if I do

> head -2 uprlwr.fmt | od -An -t oC -w10
 113 105 131 103 117 104 105 054 062 065
 054 103 012 106 111 114 105 111 104 054
 061 065 054 103 012

I pay special attention to the octal codes, looking out for any less than 40 as these are special or control characters. 012 is line-feed, 015 is carriage-return
It is possible that you need to trap for something else. Perhaps something greater than 177.

8

Hello All,

I finally managed to get fix for my issue, I am sure this is not the cleanest way to deal but still it solved our issue

here is the main logic

cat file | tr -d '\n' > temp1

This will remove all the \n from file and will generate new file with single line with ^M char. We already know that when ^M comes it's end of line so now we need to replace ^M with new line

sed 's/[^M]/\n/g' temp1 > file

This will fix the issue of removing ^M char from your file and will also remove any unwated new lines from your .cvs file on unix.

Now my line looks like following

Special Olympics Missouri,None,Muller+Company,MULLE001,4498,MS-80124-58-SOMO-ME-US-Special Olympics Basketball - St Louis,,Marc Hale,C1-MS-IS,,45305,2/18/2008,3/9/2008,N/A,MySpace ROS Leaderboard (728x90) ,cpm,1.45,"434,483",0,0,0%,0,"4739 Belleview Kansas City, KS 64112 United States",Net 30,Bill on Delivery - 3rd Party,jsydnor,N/A,Associated Profile,N/A,0,0,0,0,N/A,0.00%,N,February 2008 - February 2008,"248,276",360,0,0,N/A,0,0%,0,,N/A,N/A,N/A,"16 - 18, 18 - 24",N/A,8,N/A,United States,St.Louis-609,N/A,N/A,Feb-08,2/29/2008,2/1/2008,USD,2/18/2008,3/9/2008,,,,3832,0,452,100%,,,FIM,MySpace Standard Terms,0,,,N/A,N/A,N/A,N/A

Thanks to everyone on websites who helped me to get this fixed :slight_smile:

Cheers,
H