Sorting the csv file in Perl

Shell Programming and Scripting
1

Hi All
How all are doing today. Just struck in an issue in Perl
I have a csv file which contain 32 column, I want to make sorting in that csv file with respect to 26th column.
Is it possible to do so without any module being added?

Regards
Aditya

---------- Post updated at 10:02 AM ---------- Previous update was at 09:55 AM ----------

FYI I am usinf windows Perl

2

You could install cygwin to get the nicer sort. A CSV file with commas in the data is particularly hard to sort, even for the unix sort. I would just read the CSV with PERL libraries and put every row into a sorted container like a tree (not a hash), so I can fetch them in order.
B-Trees in Perl http://search.cpan.org/~bryce/Sort-Tree-1.09/lib/Sort/Tree.pm\#DESCRIPTION
http://search.cpan.org/~rsavage/Tree-Binary-1.05/lib/Tree/Binary.pm\#DESCRIPTION
Many Perl containers can be sorted: http://search.cpan.org/~chthorman/Data-CTable-1.03/CTable.pm\#SORTING
http://search.cpan.org/~uri/Sort-Maker-0.06/Sort/Maker.pm\#DESCRIPTION

Wading through all this, first you have to avoid one dimensional sorts unless you want to reorder your csv lines with the key first, or copy the key first as N bytes left justified so you can easily remove it. You do not need manually managed parent-child trees. Hash containers have their own hash based sort order, so when they say hash tree, the mean hash map, not tree map. I read an echo of my words here: http://www.perlmonks.org/index.pl?node_id=599204

3 
use warnings;
use strict;

my (%hash, @arr);

open FILE, "< c:/path/to/file.csv";
while (<FILE>) {
    chomp;
    @arr = split /,/;
    $hash{$arr[25]} = join(',',@arr[0..24]) . '|' . join(',',@arr[26..$#arr]);
}    
close FILE;

undef @arr;
for (sort keys %hash) {
    @arr = split /\|/, $hash{$_};
    print "$arr[0],$_,$arr[1]\n";
}
4

Check out this info:
http://www.perlmonks.org/?node_id=512942

5

A CSV file with commas in the data is particularly hard to sort, even for the unix sort, e.g.:

1,2,3,"A CSV file with commas in the data is particularly hard to sort, even for the unix sort.",5,6\r\n

Luckily, or sadly, many data sets are embedded-comma-free, lulling the unwary into forgetfulness ! Early versions of MS Access had this problem, forcing me to use tab or pipe (|) separated text.

6

@spacebar
while using sort command I am getting an error.

syntax error at C:\Users\aditya\Documents\Perl Prgms\compare.pl line 35,
near "3 -t "

---------- Post updated at 01:56 AM ---------- Previous update was at 01:50 AM ----------

@balajesuri
getting an error

Use of uninitialized value in join or string at C:\Users\aditya\Documents
\Perl Prgms\sort_check.pl line 10, <FILE> line 476.
7

Well, the link is showing shell (executable /bin/sort), not perl sort() built-in, and seems to have lost some line breaks in the data, too.

8

@adisky123: Does the csv file have 32 fields in all its rows? We can go about discussing this or you can post an exact sample of the input file.

9

@Balajesuri.. Thanks for your wonderful reply. There were some error in my file only.
But there is one more issue. The master columns on which we are performing the sorting are multiple..
I give you the example, lets suppose we have .csv file which has 4 columns
Original File

hello,code,a,check
hi,take,a,perform
tag,button,b,below
message,lap,a,hint
do,you,b,any
in,the,a,below

And we have to perform sorting based on 3rd column
So the expected output wanted

hello,code,a,check
hi,take,a,perform
message,lap,a,hint
in,the,a,below
tag,button,b,below
do,you,b,any

---------- Post updated at 01:45 PM ---------- Previous update was at 12:56 PM ----------

Hi
I tried some thing like below

use warnings;
use strict;

my (%hash, @arr, $rec);

my $p_file='sort_file.txt';
open('NEW',">$p_file") || die "Error open file $p_file\n";


open FILE, "<flat_file.csv";
while (<FILE>) {
    chomp;
    @arr = split /,/;
    $rec++;
    
    $hash{$arr[15].'|'.$rec} = join(',',@arr[0..14]) . '|' . join(',',@arr[16..$#arr]);
}    
close FILE;

undef @arr;
for (sort keys %hash) {
    @arr = split /\|/, $hash{$_};
    print NEW "$arr[0],$_,$arr[1]\n";
}

close NEW;

But with this now extra record is coming..How can i remove that..

10

You need to strip the "|$rec" component off the hash key. You can use something like this in the final for-loop: s/\|\d+$//

1 Like
11

Finally this thread ends up with this code

use warnings;
use strict;

my (%hash, @arr, $rec);

my $p_file='sort_file.csv';
open('NEW',">$p_file") || die "Error open file $p_file\n";


open FILE, "<flat_file.csv";
while (<FILE>) {
    chomp;
    @arr = split /,/;
    $rec++;
    
    $hash{$arr[15].'|'.$rec} = join(',',@arr[0..14]) . '|' . join(',',@arr[16..$#arr]);
}    
close FILE;

undef @arr;
for (sort keys %hash) {
    @arr = split /\|/, $hash{$_};
    
    @_ = split /\|/,$_;
    print NEW "$arr[0],$_[0],$arr[1]\n";
}

close NEW;

.

This is very interesting to learn Perl.
I really thanks BALAJESURI for his fabulous help. Thanks..

---------- Post updated at 02:11 PM ---------- Previous update was at 02:07 PM ----------

@balajesuri.
Can you please help me in providing remedy to this thread

12

Parsing CSV