command to remove multiple commands in particular columns

Shell Programming and Scripting
1

Hi Experts,
I actually need to remove multiple commas within the column not the entire row. Its comma delimited file
Actually the value seems to look like 1,006,000, when we open this in notepad or word pad the value look s like �1,006,000�
Actually our Sed command removes single comma and double quotes, but its not handling multiple commas , can we write something nested loop here

We have used this command swhich removes only one comma and double quotes

sed -e 's/\("[^,]*\)[,]\([^"]*"\)/\1\2/g' 
sed -e 's/"//g'

But the requirement is such we need to handle multiple commas in only particular columns in the file (Note : not all columns)
Urgent help is much appreciated
Thanks in Advance
-Babu

2

Please post sample input file and the output you are expecting. ( Including which columns you want and which you don't want to change to)

Regards
Ravi

3

Parsing CSV

4

Thanks panyam fro your quick reply!!!
here is the sample data

Input file
first,"Street1,City1,Country1",test1
second,"Street2,City2","test2"
third,"Street3,City3,Country3","A,B,C,D,E"
fourth,"A,B,C,D,E",test4"

output file

first,"Street1City1Country1",test1
second,"Street2City2","test2"
third,"Street3City3Country3","ABCDE"
fourth,"ABCDE",test4"

5 
% cat testfile                                                
first,"Street1,City1,Country1",test1
second,"Street2,City2","test2"
third,"Street3,City3,Country3","A,B,C,D,E"
fourth,"A,B,C,D,E",test4"

% cat testfile | perl -lpe 's/("[^"]+")/($temp=$1) =~ s|,||g && $temp/e'
first,"Street1City1Country1",test1
second,"Street2City2","test2"
third,"Street3City3Country3","A,B,C,D,E"
fourth,"ABCDE",test4"
6 
#  nawk -F"\"" '{OFS=FS;gsub(",","",$2);print}' infile
first,"Street1City1Country1",test1
second,"Street2City2","test2"
third,"Street3City3Country3","A,B,C,D,E"
fourth,"ABCDE",test4"

HTH

7

Hi Yazu,
Many Thanks!!!
I have a question what will be the code if i want to implement for 1, 2 and 3 columns

Appreciate it !!!

---------- Post updated at 11:02 AM ---------- Previous update was at 10:56 AM ----------

Hi Tytalus,

Many thanks for valuable input

what will be the code if i want to implement on 1, 2 and 3 column
Actually our requirement is file which has 52 columns an there we are getting 4 columns as these kind of values and we are validating only for these four

Thanks for your help once again

Apppreciate It

Babu

8 
#!/bin/sh

# rm-quotes.sh NFIELD (from 1)

NFIELD=$1

INSIDE=0

while read s; do
    for ch in `echo $s | sed 's/./& /g'`; do
        case $ch in
               ,) [ $INSIDE -eq  0 ] &&  echo , || echo ' ' ;;
              \") INSIDE=$((! $INSIDE)); echo $ch ;;
             ' ') : ;;
               *) echo $ch ;;
        esac
    done | 
        tr -d "\n" | 
        awk -F, '{ 
                    gsub(" ", "", $'$NFIELD') 
                    gsub(" ", ",")
                    print 
                 }'
done

% ./rm-quotes.sh 1 < testfile
first,"Street1,City1,Country1",test1
second,"Street2,City2","test2"
third,"Street3,City3,Country3","A,B,C,D,E"
fourth,"A,B,C,D,E",test4"

% ./rm-quotes.sh 2 < testfile
first,"Street1City1Country1",test1
second,"Street2City2","test2"
third,"Street3City3Country3","A,B,C,D,E"
fourth,"ABCDE",test4"

% ./rm-quotes.sh 3 < testfile
first,"Street1,City1,Country1",test1
second,"Street2,City2","test2"
third,"Street3,City3,Country3","ABCDE"
fourth,"A,B,C,D,E",test4"

Though it doesn't work with real csv files because of spaces (it can be fixed easy enough) and '\,' , '\"', and other issues in csv files.