Remove duplicate line detail based on column one data

Shell Programming and Scripting
1

My input file:

AVI.out     <detail>named as the RRM .</detail>
AVI.out     <detail>Contains 1 RRM .</detail>

AR0.out     <detail>named as the tellurite-resistance.</detail>

AWG.out     <detail>Contains 2 HTH .</detail>

ADV.out     <detail>named as the DENR family.</detail>
ADV.out     <detail>Contains 1 SUI1 domain.</detail>

AH7.out     <detail>Contains 1 box .</detail>
AH7.out     <detail>Contains 11 rich.</detail>

AZM.out     <detail>named as the family.</detail>

My desired output file:

AVI.out     <detail>named as the RRM .</detail>

AR0.out     <detail>named as the tellurite-resistance.</detail>

AWG.out     <detail>Contains 2 HTH .</detail>

ADV.out     <detail>named as the DENR family.</detail>

AH7.out     <detail>Contains 1 box .</detail>

AZM.out     <detail>named as the family.</detail>

I would like to remove those detail that appear at the second and keep only those detail at the first row.
Thanks for advice.

2

try this,

awk 'n!=$1{print;n=$1}' infile

regards

3

thanks gaurav1086,
your awk code work perfectly for my problem ^^
thanks again :slight_smile:

4

hello ,

your are most welcome.

Regards.

5

Try:

awk '!NF || ++A[$1]==1'  < file
6

thanks dennis,
your awk code is another way to solve my problem as well ^^
thanks a lot.

7

this is better, which can remove duplicated empty line.

8

Hi gaurav1086,
Can I ask you what is the back story of your awk code?
I not very understand that why the awk code able to print out my desired output result.
Thanks again.

9

Hello patrick,

awk 'n!=$1{print;n=$1}' infile

I have implemented the algorithm in the following way:
.if the first field is NOT equal to my stored field(i.e. n)
then 1.print the record.
2.update the stored field.
Otherwise do not do anything.
I have explained it in the most basic way.
Hope you understand. Feel free to ask any questions about it.
Regards.
Gaurav.

10

thanks for your explanation, gaurav1086 :slight_smile:
I understand what is the reason of your awk code now ^^
I'm very appreciate it.
Thanks again for your explanation.

11

Hello patrick,

I am glad that I was able to help you.
you are most welcome.

Take care,
Regards.
Gaurav.