Troubleshoot awk code that fixes line breaks

UNIX for Beginners Q & A
1

Hi. I'm new to UNIX. I've searched this forum and found an example of awk that works for my file to fix line breaks.

awk -F'|' 'NF != 48 || !$48 {printf $0; getline} 1' file1.csv > file2.csv

However, sometimes the line is broken across three lines. But if I execute this again, it fixes that broken line but the last record is repeated two or three times depending on how many times I run it.

1) How do I execute it once and get it to fix all the broken lines, regardless of how many lines the record takes up?
2) How do I get it to stop repeating the last record?

2

How about (untested)

 while (NF < 48) {getline X; $0 = $0 X}

Please add a reasonable error checking, as this doesn't account for e.g. NF never reaching 48.

3

That code alone gives a syntax error. I don't know how to write it correctly yet.

4

Please post a representative sample.

5

A sample of the file?

6

Yes, please.

7

Below is the header and a few records with one record in the middle that the user has used the enter key on the vendor form, causing it to go over three lines. In this example, all records begin with ,"abc". (Data is not real to protect sensitive information.)

"Account Name","Community Name","Last Name","First Name","ID_number","Admin Gift Pledge ID","Admin Gift Source","Donor Instructions","Gift Other Designation","Matching Gift Company old","Tribute Name","Tribute Notification","Tribute Type","Tribute ID Number","Premiums","Admin Gift Method","Gift Admin Initials","Campaign","Campaign ID","Order Total","Transaction ID","Purchase Date","Authorization Code","Customer Trans Number","Retrieval Code","Appeal Code","From Facebook","Billing Name","Credit Card Type","Billing Email","Billing Phone","Billing Street 1","Billing Street 2","Billing City","Billing State","Billing Zip","Billing Country","Scheduled Payments Active","Payment Frequency","Total Scheduled Amount","Next Payment Amount","Next Payment Date","Payments Remaining","Payment Type","Amount To Date","First Payment Date","Original Transaction Id","Event Name"

,"abc","Anderson","Tom",,,,," Gift for Instruction",,"Mickey","mm@att.net","in honor of someone special",,,,,"API Make a Gift","7440","500.00","56370","1/15/2016 11:29:52 AM","04442D","20160115000004",,"13529","False","Tom Anderson","VISA","tm@hotmail.com",,"1234 S. Dorchester",,"Chicago","IL","60637","US",,,,,,,"OT",,,,
,"abc","Town","Rob",,,,"The Town Family
1234 Grovecrest Cv.
Germantown, TN 38139","Big Event",,"Ray E Town",,"in memory of someone special",,,,,"Make a Gift","4045","50.00","56357","1/15/2016 7:25:48 AM","01530P","20160115000005",,"10063","False","Rob Town","Disc","rob.town@gmail.com",,"2693 Saps Ct.","","Burlington","KY","41005","US",,,,,,,"OT",,,,
,"abc","Wills","Alaina","000022333",,,,,,,,,,,,,"Donation Form","4676","15.00","56354","1/15/2016 7:07:29 AM","0332B","20160115000004",,,"False","Alaina Wills","Mast","alaina.wills@gmail.com","4444449986","64 central blvd apt 2","","Whitestown","IN","46075","US","True","12",,"15.00","2/15/2016 5:06:27 PM",,"PT","45.00","10/23/2015 5:06:27 PM","45989",
,"abc","Walsh","Tim","0000123400",,,,,,,,,,,,,"Make a Gift","4045","90.00","56316","1/15/2016 7:05:08 AM","0z098","20160000000",,,"False","Tim Walsh","Disc","tw@hotmail.com","","5823 Gram Court","","Indianapolis","IN","46250","US","True","12",,"90.00","2/15/2016 1:30:01 PM",,"PT","1170.00","1/15/2015 1:30:01 PM","18932",

8

Please use code tags as required by forum rules!

You didn't mention the fields are enclosed in double quotes which might need additional action. Howsoever, how about

awk -F, '{while (NF < 48) {getline X; $0 = $0 X}}1' file
1 Like
9

I was close. I didn't use a comma after -F and left the | in.
Thank you so much! It worked like charm! You're awesome!