Remove repeated line using Perl

Shell Programming and Scripting
1

I am new to Perl and in text file of around 1000 lines having around 500 repeated line which I felt is no use and want to remove these line.so can somebody help in same for providing sample code how can i remove these repeated line in a file.

2 
sort -u
3

here is one way to do it Remove Duplicate Lines from a File - Perl

do you have to use Perl? If not use the method that rdcwayx suggests.

4

yes i have to use perl script.
so can somebody mail me sample code of same.

5

Try

perl -i -ne 'next if $s{$_}++; print;' file
6

After googling I found script for removing repeated line in a file.

next if $sLine =~ m/^\s*$/;  
$sLine=~s/^\s+//;
 $sLine=~s/\s+$//;  
print OUT qq{$sLine\n} unless ($hTmp{$sLine}++);

Basically regular expression are used to remove repeated line now Point is am I am not getting clear idea about these how they are working.Can somebody provide me some sort of help in that.

7

The first three lines disregard (and do **not** print) blank lines and trim non-blank lines.
The fourth line is just another way of writing Klashxx's one-liner.
Try both of them with sample data and it should make things clearer.

tyler_durden

8

It's state it will print only if line is not repeated.
But why unless statement is used further.

9

I hope you do realize that you've answered your own question.
(Hint) => unless is the opposite of if.

tyler_durden

10

Thanks..I got it.