Need help in filters

Shell Programming and Scripting
1

Hi,

I have input data.
9214919702; B5; 1;20070216;
9231590437; BY; 1;20070215;9;20091022;12;20091022;
9211765888; AZ; 1;20080802;1;20080802;14;20091027;
9231592590; BY; 1;20070215;9;20091026;9;20091026;
9252412219; MM; 1;20070217;
9214917135; MM; 1;20070215;
9214917056; B5; 1;20070215;
9219381040; E2; 1;20070215;
9253112376; NM; 1;20071222;2;20071222;9;20091030;12;20091030;

Can I get output like those rows have duplicated columns (red color)

output required
9211765888; AZ; 1;20080802;1;20080802;14;20091027;
9231592590; BY; 1;20070215;9;20091026;9;20091026;

Thanks in Advance.
Suresh

2 
$
$ cat f1
9214919702; B5; 1;20070216;
9231590437; BY; 1;20070215;9;20091022;12;20091022;
9211765888; AZ; 1;20080802;1;20080802;14;20091027;
9231592590; BY; 1;20070215;9;20091026;9;20091026;
9252412219; MM; 1;20070217;
9214917135; MM; 1;20070215;
9214917056; B5; 1;20070215;
9219381040; E2; 1;20070215;
9253112376; NM; 1;20071222;2;20071222;9;20091030;12;20091030;
$
$ perl -F";" -lane '$F[2]=~s/^\s+//;
                    print if (("$F[2]:$F[3]" eq "$F[4]:$F[5]" || "$F[4]:$F[5]" eq "$F[6]:$F[7]") && ($F[4] ne ""))' f1
9211765888; AZ; 1;20080802;1;20080802;14;20091027;
9231592590; BY; 1;20070215;9;20091026;9;20091026;
$
$

tyler_durden

3 
while(<DATA>){
	my @tmp=$_=~/([0-9];[0-9]+)/g;
	my %hash = map {$_,1} @tmp;
	my @t=keys %hash;
	print if $#tmp != $#t;
}
__DATA__
9214919702; B5; 1;20070216;
9231590437; BY; 1;20070215;9;20091022;12;20091022;
9211765888; AZ; 1;20080802;1;20080802;14;20091027;
9231592590; BY; 1;20070215;9;20091026;9;20091026;
9252412219; MM; 1;20070217;
9214917135; MM; 1;20070215;
9214917056; B5; 1;20070215;
9219381040; E2; 1;20070215;
9253112376; NM; 1;20071222;2;20071222;9;20091030;12;20091030;
4 
nawk ' {
        ln = $0
        gsub(" ", "")
        for( i =1 ; i <= NF ; i++ ){
                if( ++f[$i] > 1 && f[$(i - 1)] > 1 )                              
                        printf("%s\n", ln)
        }
        split("", f)
} ' FS=";" OFS=";"
5

Hi Durden,
I have tried below command, working very fine. But I have columns more than 20. then what is the value in -- $F[4] ne ""))

$ perl -F";" -lane '$F[2]=~s/^\s+//;
print if (("$F[2]:$F[3]" eq "$F[4]:$F[5]" || "$F[4]:$F[5]" eq "$F[6]:$F[7]") && ($F[4] ne ""))' f1

---------- Post updated 11-09-09 at 11:11 PM ---------- Previous update was 11-08-09 at 11:12 PM ----------

Anybody ... plz help

6

Sorry this line is not clear.
Post your actual input data and expected output data.

tyler_durden

7

Hi Durden,

Input file data is like

9211765888; AZ; 1;20080802;1;20080802;14;20091027;
9231592590; BY; 1;20070215;9;20091026;9;20091027;
9252412219; MM; 1;20070217;
9211765888; AZ; 1;20080802;1;20080802;2;20080909;2;20080909;4;20070809;5;20091001;5;20091001;12;20080509;13;20090101;14;20091027;14;20091027;
9231592590; BY; 1;20070215;9;20091026;9;20091026;20;20091010;20;20091111

expected OUTPUT - only these rows have duplicate columns.
9211765888; AZ; 1;20080802;1;20080802;14;20091027;
9211765888; AZ; 1;20080802;1;20080802;2;20080909;2;20080909;4;20070809;5;20091001;5;20091001;12;20080509;13;20090101;14;20091027;14;20091027;
9231592590; BY; 1;20070215;9;20091026;9;20091026;20;20091010;20;20091111

8 
#!/usr/bin/perl

use strict;
use warnings;

while(my $line = <>){
        chomp $line;
        $line =~ s/\s+//g;
        my @cols = split(';',$line);
        my %hash;
        for(my $i=0;$i<=$#cols;$i++){
                if (exists $hash{$cols[$i]} && $hash{$cols[$i]} eq $cols[$i+1]){
                        print "$line is a repeat \n";
                        last;
                }
                else {
                        $hash{$cols[$i]} = $cols[$i+1];
                         $i++;
                }
        }
}

the script can be called as

cat abc.txt | perl scriptname.pl

or it can be copied on the command line as well.

HTH,
PL

9

Hi Daptal

gettting error

ksh: use: not found.
ksh: 0403-057 Syntax error: `)' is not expected.