Hi I have below pattern
A: Apple 2 B:Bolls 4 total_count = 6
A: pens 4 B:Bags 4 total count = 8
A: pens 4 B:Bags 4
A: cells 6
A: jobs 6
Output I need :
A: Apple 2 B:Bolls 4 total_count = 6
A: pens 4 B:Bags 4 total count = 8
A: cells 6
A: jobs 6
2nd line and 3rd lines of 6 columns are same . So I need to remove 3rd line.(which all 6 columns are same)
Any help ??
Thanks for advance
Your statements are inconsistent.
The title of this thread is UNIX sort which lead me to believe that you wanted to use the sort command's -u option to discard all but one line for each set of lines with duplicate keys.
But sort uses a single character as a field separator (except for the default case which uses sequences of one or more spaces and tabs as field separators). So, with your input, the first six columns (or fields) in input line 2 are:
A: pens 4 B:Bags 4 total
and there are only five columns in input line 3. If you meant that you wanted to sort on five fields (instead of six), the output order would be very different from what you said you want and the standards don't specify whether you would get line 2 or line 3 as the output from those two lines. Assuming that your text to be processed is in a file named file , the command:
sort -u -k1,5 file
would produce the output:
A: Apple 2 B:Bolls 4 total_count = 6
A: cells 6
A: jobs 6
A: pens 4 B:Bags 4 total count = 8
or the output:
A: Apple 2 B:Bolls 4 total_count = 6
A: cells 6
A: jobs 6
A: pens 4 B:Bags 4
If this isn't the output you were trying to get, please clearly explain what you are trying to do.
Sorry for the confusion.
I am trying to use unix sort on below file which contains
A: Apple 2 B:Bolls 4 total_count = 6
A: pens 4 B:Bags 4
A: pens 4 B:Bags 4 total_count = 8
A: pens 4 B:Bags 4
A: pens 4 B:Bags 4 total_count = 8
A: cells 6
A: jobs 6
Output which I am expecting is :
A: Apple 2 B:Bolls 4 total_count = 6
A: pens 4 B:Bags 4 total_count = 8
A: cells 6
A: jobs 6
I tried
sort -u -k1,5
output is
A: Apple 2 B:Bolls 4 total_count = 6
A: cells 6
A: jobs 6
A: pens 4 B:Bags 4
Output which I am expecting is
A: Apple 2 B:Bolls 4 total_count = 6
A: pens 4 B:Bags 4 total_count = 8
A: cells 6
A: jobs 6
I don't see, how sort alone could solve this. Because it would need to decide to discard:
A: pens 4 B:Bags 4
while keeping
A: pens 4 B:Bags 4 total_count = 8
--
Could you try if this is closer to what you need:
awk 'NF!=5' infile | uniq
if the file is already sorted on field number 3, or otherwise..
awk 'NF!=5' infile | sort -u -k3,3n -k1,2 -k4
It's working for above code and trying to implement same on below pattern
A: Apple 2 B:Bolls 4 total_count = 6
A: pens 4
A: pens 4
A: pens 4 B:Bags 4 total_count = 8
A: pens 4
A: pens 4 B:Bags 4 total_count = 8
A: cells 6
A: jobs 6
output
A: cells 6
A: pens 4
A: pens 4 B:Bags 4 total_count = 8
A: Apple 2 B:Bolls 4 total_count = 6
A: jobs 6
But I am expecting 2nd line should not come as a output. it means I want output like below
A: pens 4 B:Bags 4 total_count = 8
A: Apple 2 B:Bolls 4 total_count = 6
A: cells 6
A: jobs 6
Thanks Advance
So, at first you said you wanted to match on 6 fields. Then you wanted to match on 5 fields. And, now are you saying you only want to match on the 1st 3 fields? And:
A: jobs 6
is somehow magically dropped from the output?
Please explain in English what determines what lines are to be ignored, whether or not lines are considered to be a match, and when there are lines that match, which of the matching lines are to be printed.
Does the output order matter? If it does, please explain in English what the sort keys are.
I want to ignore the specific lines which are matching 3 fields means
A: pens 4
A: pens 4 B:Bags 4 total_count = 8
A: cells 6
A: jobs 6
above code line 1 and line 2 of 3 fields are same. When ever I saw this kind of pattern I want to remove a line which is having 3 fields are same. In the above example I want to remove a 1st line.
output which I am expecting
A: pens 4 B:Bags 4 total_count = 8
A: cells 6
A: jobs 6
You keep giving us examples of what you want without ever specifying the algorithm that should be used to determine which input rows should be written to the output and which input rows should be discarded. Despite repeatedly saying that you want to use the sort utility, one common thing in all of your sample desired outputs is that none of the output you say you want is sorted.
If you're willing to use awk instead of sort and your unstated algorithm is something like:
For each set of lines where the 1st three fields are identical:
then the following may do what you want:
awk '
lfc[$1, $2, $3] < NF {
lfc[$1, $2, $3] = NF
l[$1, $2, $3] = $0
}
END { for(k in l)
print l[k]
}' file
If you want to try this on a Solaris/SunOS system, change awk to /usr/xpg4/bin/awk , /usr/xpg6/bin/awk , or nawk .
If file contains (as in your 1st example):
A: Apple 2 B:Bolls 4 total_count = 6
A: pens 4 B:Bags 4
A: pens 4 B:Bags 4 total_count = 8
A: pens 4 B:Bags 4
A: pens 4 B:Bags 4 total_count = 8
A: cells 6
A: jobs 6
the output produced is:
A: jobs 6
A: cells 6
A: Apple 2 B:Bolls 4 total_count = 6
A: pens 4 B:Bags 4 total_count = 8
If file contains (as in your 2nd example):
A: Apple 2 B:Bolls 4 total_count = 6
A: pens 4 B:Bags 4
A: pens 4 B:Bags 4 total count = 8
A: cells 6
A: jobs 6
the output produced is:
A: jobs 6
A: cells 6
A: Apple 2 B:Bolls 4 total_count = 6
A: pens 4 B:Bags 4 total count = 8
And, if file contains (as in your 3rd example):
A: pens 4
A: pens 4 B:Bags 4 total_count = 8
A: cells 6
A: jobs 6
the output produced is:
A: jobs 6
A: cells 6
A: pens 4 B:Bags 4 total_count = 8
Is this what you're trying to do?