How to parse csv format?

Shell Programming and Scripting
1

Hi,
I have a file with 3 fields in csv format:

/tmp/foo/,MODIFY,bar/toto
"/tmp/foo, bar/","ATTRIB,ISDIR","toto, tata/foobar"

I would like to split fields in order to obtain the following:
Line1:

/tmp/foo/
MODIFY
bar/toto

Line2:

/tmp/foo, bar/
ATTRIB,ISDIR
toto, tata/foobar

Can't find my way through it!
How would you do that?
Thanks for your help.
Santiago

2

Hi, Try This

#!/usr/bin/perl


    use Text::CSV;

    my $file = 'inpit.csv';

    my $csv = Text::CSV->new();

    open (CSV, "<", $file) or die $!;

    while (<CSV>) {
       chomp;
        if ($csv->parse($_)) {
            my @columns = $csv->fields();
            $i=0;
            print "\n\nLINE $. \n";
            while (<@columns>) {
            print "$columns[$i] \n";
            $i = $i + 1 ;
            }
        } else {
            my $err = $csv->error_input;
            print "Failed to parse line: $err";
        }
    }
    close CSV;
3 
awk '{if ($0~/"/) {split($0,a,"\",\"")} else {split($0,a,",")}}
     {for (i=1;i<=length(a);i++) {sub(/"/,"",a);print a}}' urfile
4

Thanks pravin27,
I forgot to say that I'm trying to find a shell language solution.
Altough your approach is interesting, I can't make it work. Do I need to do something special?

Can't locate Text/CSV.pm in @INC (@INC contains: /etc/perl /usr/local/lib/perl/5.10.0 /usr/local/share/perl/5.10.0 /usr/lib/perl5 /usr/share/perl5 /usr/lib/perl/5.10 /usr/share/perl/5.10 /usr/local/lib/site_perl .) at parser.pl line 4.
BEGIN failed--compilation aborted at parser.pl line 4.

Thanks rdcwayx,
I like your awk approach because it can fit my needs. But I get the following error:

awk: line 2: illegal reference to array a

What did I do wrong?

---------- Post updated at 22:29 ---------- Previous update was at 19:33 ----------

Hi there,
I did it another way (inspired by rdcwayx's idea) with sed:
The whole idea is to be able to interpret the csv output of inotifywait.

inotifywait -mrc -e modify -e attrib -e moved_to -e create /tmp/ 2>"$inotify_err" | while read -r; do
	eval "$(echo "$REPLY" | sed -r 's#'\''#'\'\\\\\'\''#; s#^("([^"]*)"|([^"][^,]*)),("([^"]*)"|([^"][^,]*)),("([^"]*)"|([^"][^,]*))$#file_name='\''\2\3\8\9'\''; action='\''\5\6'\''#')"
	echo "fn=$file_name"
	echo "a=$action"
done

Thanks for your help
Santiago

5

I test again, and seems fine.

$ awk '{if ($0~/"/) {split($0,a,"\",\"")} else {split($0,a,",")}}
>      {for (i=1;i<=length(a);i++) {sub(/"/,"",a);print a}}' urfile
/tmp/foo/
MODIFY
bar/toto
/tmp/foo, bar/
ATTRIB,ISDIR
toto, tata/foobar

Maybe your system is solaris, you can use nawk.

6

Thanks rdcwayx for your interest.
My system is debian. Here is my trace:

santiago@titan:~$ cat urfile
/tmp/foo/,MODIFY,bar/toto
"/tmp/foo, bar/","ATTRIB,ISDIR","toto, tata/foobar"
"/tmp/foo, bar/",ATTRIB,"toto, tata/foobar"
santiago@titan:~$ awk '{if ($0~/"/) {split($0,a,"\",\"")} else {split($0,a,",")}} {for (i=1;i<=length(a);i++) {sub(/"/,"",a);print a}}' urfile
awk: line 1: illegal reference to array a
santiago@titan:~$ nawk '{if ($0~/"/) {split($0,a,"\",\"")} else {split($0,a,",")}} {for (i=1;i<=length(a);i++) {sub(/"/,"",a);print a}}' urfile
nawk: line 1: illegal reference to array a

How can I get the version of awk I'm using?

7

Hi,
you have install the attached module.
1) Save this tar file in /tmp/Text-CSV-1.18.tar
2) Untar the file.
3) INSTALLATION
Go to the directory "/tmp/Text-CSV-1.18"
To install this module type the following:

perl Makefile.PL
make
make test
make install

8

Looks almost like "How can I use X to do Y?". Here's Z: why not use the --format switch of inotifywait to define your own format, eg with a semicolon as delimiter?

Installation of Perl modules BTW:

perl -MCPAN -e 'install Text::CSV'
9

Hi Pludi,

Thanks for the information.

10

Thx pludi.
This is really embarasing now. I don't know if I started my project with another version of inotify but I completely missed that option.
Thank you very much!

11

Hi.

The module Text::CSV is available in a Debian repository.

Many people like to to use the package manager to install items. With apt-get using the environment:

OS, ker|rel, machine: Linux, 2.6.26-2-amd64, x86_64
Distribution        : Debian GNU/Linux 5.0

here's a simulated install:

$  apt-get -s install libtext-csv-perl
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following extra packages will be installed:
  libtext-csv-xs-perl
The following NEW packages will be installed:
  libtext-csv-perl libtext-csv-xs-perl
( ... )
Inst libtext-csv-perl (1.06-1 Debian:5.0.5/stable)
Inst libtext-csv-xs-perl (0.52-1 Debian:5.0.5/stable)
Conf libtext-csv-perl (1.06-1 Debian:5.0.5/stable)
Conf libtext-csv-xs-perl (0.52-1 Debian:5.0.5/stable)

Best wishes ... cheers, drl

12

I also faced the same problem of "illegal reference to array a" when I run the command

"awk '{if ($0~/"/) {split($0,a,"\",\"")} else {split($0,a,",")}} {for (i=1;i<=length(a);i++) {sub(/"/,"",a);print a}}' urfile

Then later I modified the above command a bit and it is working fine. The command & output is as below:

bash-3.00$ cat urfile
/tmp/foo/,MODIFY,bar/toto
"/tmp/foo, bar/","ATTRIB,ISDIR","toto, tata/foobar"

bash-3.00$ awk '{a[0]=""; if($0~/^"/) {FS="\",\""; split($0,a,"\",\"");} else {FS=",";split($0,a,",");}} { for(x=1;x<=NF;x++) {sub(/"/,"",$x); print $x;}}'  urfile
/tmp/foo/
MODIFY
bar/toto
/tmp/foo, bar/
ATTRIB,ISDIR
toto, tata/foobar

Hope this helps. Thanks.