Removing newline characters within DEL quotes.

Shell Programming and Scripting
1

Hi,

Text file has DEL character(ASCII code 127) as quotes with comma as field delimiter. If any of the field contains new line character then I need to remove it. Please help me to achieve this.

Thanks
Vikram

2

Samples, please (in code tags). Any attempts from your side?

3

Firstly creating a test input file. I used this method:

$ DEL=$(printf '%b' "\x7F")
$ sed "s/~/$DEL/g" > infile <<EOF
> one,two,~Multi-line
> field
> with embedded ,~,four,five
> one,two,three,four,five
> one,two,~simple,string~,four,five
> EOF
$ sed "s/$DEL/|/g" infile
one,two,|Multi-line
field
with embedded ,|,four,five
one,two,three,four,five
one,two,|simple string|,four,five

And here is a GNU awk solution using a modified version of Lorance Freeshell's AWK CSV parser, Lorance hasn't updated this parser since 2009 and it has a few bugs corrected in the attached version:

awk -v DEL=$(printf "%b" "\x7F") '
@include "csv.awk"
{ num_fields = csv_parse($0, csv, ",", DEL, "", "\n", 1)
  for(i=1;i<=num_fields;i++) gsub("\n", " ", csv);
  print csv_create(csv, num_fields, ",", DEL)
}' infile > outfile

If you don't have GNU awk you can insert the csv.awk file in the code directly instead of using the GNU @include feature used above.

Result:

$ sed "s/$DEL/|/g" outfile
one,two,|Multi-line field with embedded ,|,four,five
one,two,three,four,five
one,two,simple string,four,five
4

Hi,

Thanks for for your help.

I found this code to remove the new line characters present within the quotes.

awk '(NR-1)%2  {$1=$1}1' RS=\" ORS=\" infile

This is working fine. But is adding extra line with " in it. Please help to remove the last line with ".

 
 
Sample Data :
 
"Line1  RENT-A-
CAR XYZ LTD","00N0H","Enterprise Lake
View Way"
"Line 2 RENT-A-
CAR XYZ LTD","00N0H","Enterprise Lake
View Way"
 
Result :
 
"Line1 RENT-A- CAR XYZ LTD","00N0H","Enterprise Lake View Way"
"Line 2 RENT-A- CAR XYZ LTD","00N0H","Enterprise Lake View Way"
"
 
Expected:
 
"Line1 RENT-A- CAR XYZ LTD","00N0H","Enterprise Lake View Way"
"Line 2 RENT-A- CAR XYZ LTD","00N0H","Enterprise Lake View Way"

Thanks
Vikram

5

try this:

awk '{for(i=1;i<=NF;i+=6){print $i,$(i+1),$(i+2),FS,$(i+3),$(i+4),$(i+5)}}' FS="\n"  RS=""