Reform Lines in File without blank lines and spaces

Shell Programming and Scripting
1

Hello All,
I have a file with data as below. Each line consists of 21 fields. I am not able to load them back to the database.

50733339,"834","834            ","005010X279A1","N","Y","007977163","0001      ",30,"2110D ","EB   ","EB007 ","2    ","Conditional Required Data Element Miss
ing
",,"REJ            ",,"8    ","Segment has Data Element Errors
","2012-03-03-00.07.48.074377",
50732730,"834","834            ","005010X279A1","N","Y","007977108","0001      ",14,"2100C ","DTP  ","DTP002","1    ","Required Data Element Missing
",,"REJ            ",,"8    ","Segment has Data Element Errors
","2012-03-03-00.00.37.832866",

I need help to reform the lines in the files as below (remove all leading and trailing spaces and reform the 21 fields into a single line. I have marked the lines in different colors for clear understanding.

50733339,"834","834","005010X279A1","N","Y","007977163","0001",30,"2110D ","EB","EB007 ","2","Conditional Required Data Element Missing",,"REJ",,"8    ","Segment has Data Element Errors","2012-03-03-00.07.48.074377",
50732730,"834","834","005010X279A1","N","Y","007977108","0001",14,"2100C ","DTP","DTP002","1","Required Data Element Missing",,"REJ",,"8","Segment has Data Element Errors","2012-03-03-00.00.37.832866",

Any assistance to achieve the above would be very very helpful

2

Use:

perl -pi -e "s/([^,])\s*\n/\1/g" infile
1 Like
3

Need a bit more of help over this one. The command is not working for some of the lines. Can you please help if the lines are as below:

54961242,"780","780-HORIZ-00000","005010X223A2","Y","N","000001403","1178      ",30,"2300  ","HI   ",,,,,"REJ            ","PNASC.HIPDT.NARJ6NNJ.NEWA.D120309.T232827.0035430-H","8    ","Segment has Data Element Errors
","2012-03-10-00.07.41.438918","NM1*IL*1*FELDER*PAMELA****MI*YHQ3HZN39529840
CLM*HAH53954*4890.13***13:A:1**A*Y*Y
HI*BF:4019:::::::Y*BF:V5869:::::::E"
54961243,"780","780-HORIZ-00000","005010X223A2","Y","N","000001403","1185      ",36,"2300  ","HI   ",,,,,"REJ            ","PNASC.HIPDT.NARJ6NNJ.NEWA.D120309.T232827.0035430-H","8    ","Segment has Data Element Errors
","2012-03-10-00.07.41.499676","NM1*IL*1*MCCLURKIN*STEPHANIE****MI*NJX3HZN14544770
NM1*QC*1*MCCLURKIN*WILLIAM"

I need them as below:

54961242,"780","780-HORIZ-00000","005010X223A2","Y","N","000001403","1178",30,"2300","HI",,,,,"REJ","PNASC.HIPDT.NARJ6NNJ.NEWA.D120309.T232827.0035430-H","8","Segment has Data Element Errors","2012-03-10-00.07.41.438918", "NM1*IL*1*FELDER*PAMELA****MI*YHQ3HZN39529840CLM*HAH53954*4890.13***13:A:1**A*Y*YHI*BF:4019:::::::Y*BF:V5869:::::::E"
54961243,"780","780-HORIZ-00000","005010X223A2","Y","N","000001403","1185",36,"2300","HI",,,,,"REJ","PNASC.HIPDT.NARJ6NNJ.NEWA.D120309.T232827.0035430-H","8    ","Segment has Data Element Errors","2012-03-10-00.07.41.499676","NM1*IL*1*MCCLURKIN*STEPHANIE****MI*NJX3HZN14544770NM1*QC*1*MCCLURKIN*WILLIAM"

The data in bold represents the beginning of each line. Thanks a lot in advance

4

This should be fine for the sample provided:

perl -pi -e 's/([^",])\s*\n/\1/g' infile