Duplicate records

Shell Programming and Scripting
1

Gents,

Please give a help

file

--BAD STATUS NOT RESHOOTED--
 *** VP 41255/51341 in sw 2973
 *** VP 41679/51521 in sw 2973
 *** VP 41687/51653 in sw 2973
 *** VP 41719/51629 in sw 2976
--BAD COG NOT RESHOOTED--
 *** VP 41689/51497 in sw 2974
 *** VP 41699/51677 in sw 2974
 *** VP 41297/51689 in sw 2977
 *** VP 41699/51437 in sw 2974
 *** VP 41699/51605 in sw 2974
 *** VP 41677/51449 in sw 2973
 *** VP 41689/51389 in sw 2974
--Vps Rejected or Missed--
 *** VP 41255/51341 in sw 2973
 *** VP 41677/51449 in sw 2973
 *** VP 41679/51521 in sw 2973
 *** VP 41687/51653 in sw 2973
 *** VP 41259/50417 in sw 2974
 *** VP 41689/51389 in sw 2974
 *** VP 41689/51497 in sw 2974
 *** VP 41699/51437 in sw 2974
 *** VP 41699/51605 in sw 2974
 *** VP 41699/51677 in sw 2974

I try

awk 'FNR == NR {c[substr($0,9,11)]++;next}c[substr($0,9,11)] -- == 1' file file

got this

--BAD STATUS NOT RESHOOTED--
 *** VP 41719/51629 in sw 2976
--BAD COG NOT RESHOOTED--
 *** VP 41297/51689 in sw 2977
--Vps Rejected or Missed--
 *** VP 41255/51341 in sw 2973
 *** VP 41677/51449 in sw 2973
 *** VP 41679/51521 in sw 2973
 *** VP 41687/51653 in sw 2973
 *** VP 41259/50417 in sw 2974
 *** VP 41689/51389 in sw 2974
 *** VP 41689/51497 in sw 2974
 *** VP 41699/51437 in sw 2974
 *** VP 41699/51605 in sw 2974
 *** VP 41699/51677 in sw 2974

But i will like to get this output

--BAD STATUS NOT RESHOOTED--
 *** VP 41255/51341 in sw 2973
 *** VP 41679/51521 in sw 2973
 *** VP 41687/51653 in sw 2973
 *** VP 41719/51629 in sw 2976
--BAD COG NOT RESHOOTED--
 *** VP 41689/51497 in sw 2974
 *** VP 41699/51677 in sw 2974
 *** VP 41297/51689 in sw 2977
 *** VP 41699/51437 in sw 2974
 *** VP 41699/51605 in sw 2974
 *** VP 41677/51449 in sw 2973
 *** VP 41689/51389 in sw 2974
--Vps Rejected or Missed--
 *** VP 41259/50417 in sw 2974

Thanks for your help

2

Not clear. What do you want to achieve?

3

---------- Post updated at 11:36 AM ---------- Previous update was at 11:27 AM ----------

RudiC

I want to delete duplicate values keeping firts entry..

4

Hi, try:

awk '!A[substr($0,9,11)]++' file

output:

--BAD STATUS NOT RESHOOTED--
 *** VP 41255/51341 in sw 2973
 *** VP 41679/51521 in sw 2973
 *** VP 41687/51653 in sw 2973
 *** VP 41719/51629 in sw 2976
--BAD COG NOT RESHOOTED--
 *** VP 41689/51497 in sw 2974
 *** VP 41699/51677 in sw 2974
 *** VP 41297/51689 in sw 2977
 *** VP 41699/51437 in sw 2974
 *** VP 41699/51605 in sw 2974
 *** VP 41677/51449 in sw 2973
 *** VP 41689/51389 in sw 2974
--Vps Rejected or Missed--
 *** VP 41259/50417 in sw 2974
1 Like
5

I will.like to get the same fil� as desired please

6

The output is exactly what you asked for, word-for-word, letter-for-letter.

If that's not the output you want, please tell us what you do want.

1 Like
7

I see that you and Scrutinizer edited your posts within 2 minutes of each other, so there may be some confusion, could you give it another look please?

8

I have a problem (msg)
[substr: Event not found

9

That can't happen in single quotes. You must have changed them to double quotes. Don't do that. Use the code as given.

1 Like
10

looks like this is eeror of bash version..... 4.2.37(1)

11

not likely. Please post exactly what command you're running....

1 Like
12 
awk '!A[substr($0,9,11)]++' file
13

interesting... running with bash 4.2.46(1) under centos it does work exactly as desired....
Once again, are you sure you're using SINGLE-quotes and not DOUBLE-quotes?
What OS is it for? What version of awk?
Also please do echo $0 in the same shell you're running the awk code in and post it here

1 Like
14

yes i am using single-quotes..
OS debian 7
mawk 1.3.3 Nov 1996

15

weird...
and what's the output of echo $0 in the same shell?

1 Like
16

give msg

-csh
17

ok, you're inside the csh , not bash - this still not explaining why you're getting this error using single-quotes.
try going into bash (typing bash at you shell prompt) and re-run awk

1 Like
18

yes, it works..
But if there is the possibility to generate other code to work with csh ... because i have other scripts working and i will like to adapt the code on one of my scripts.

thanks a lot

19 
awk '\!A[substr($0,9,11)]++' file

But we really, really recommend you break yourself of the csh habit as soon as feasible. It has unresolvable design flaws and is pretty much depreciated now. Even it's inventor doesn't retain much fondness for it anymore.

2 Likes