Search and remove the lines

Shell Programming and Scripting
1

Hallo Team,

Hope you are having a wonderful Friday. Here goes i am searching for a pattern and after finding the lines which contain this pattern i want to remove/delete them. This is my code:

 grep Originating BW*2013*|grep -v "ACCOUNT NOT FOUND"|grep -v "Unknown called number"|grep -v "Y,Unknown called on-net party"|grep -v "|Y,Unknown calling on-net party"|grep -v "Y,ACCOUNT IN RECALC."|grep "|Y,Unclear scenario:On-net to Off-net PGW, On-net to Off-net Partner IP:BSO,PGWO"

My output is like below and i would like to delete the lines or files:

BW-CDR-20130711180000-2-AC162DB137C8-037434-000.csv_rejected_4213307-20130718193659.log:0072782780AC162DB137C820130711155727.3020+020000,MRPRICE,Normal,+27313671590,,Originating,+27313671590,Public,0218369379,20130711155727.302,0+020000,Yes,20130711155758.255,20130711155803.894,016,VoIP,,0218369379,interlat,to,0218369379,,remote,10.0.108.11:5060,BW175727304110713-2083830430@196.35.130.5,G729/8000/1,10.0.12.58,4864d0100007f13ce000a4f4b00027100@127.0.0.1,,,,mrp_headoffice-cco,,,,,,,,,,y,public,,832627663:2227,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,0313671590@isvoip.net,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,34.717,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,Network,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,|Y,Unclear scenario:On-net to Off-net PGW, On-net to Off-net Partner IP:BSO,PGWO
BW-CDR-20130712074500-2-AC162DB137C8-037599-000.csv_rejected_4214106-20130718193659.log:0072880124AC162DB137C820130712053952.2820+020000,NORTON,Normal,+27110170600,,Originating,+27110170600,Anonymous,0218026894,20130712053952.282,0+020000,Yes,20130712054019.351,20130712054019.415,016,VoIP,,0218026894,interlat,to,0218026894,,local,10.0.108.11:5060,BW073952284120713166667830@196.35.130.5,G729/8000/1,10.0.4.117,B27FAE28-E9EC11E2-B056A9B6-23934ADB@196.38.51.242,,,,NORTON-HO,,,,,,,,,,y,public,,833851331:2448,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,0110170600@isvoip.net,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,26.123,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,Network,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,|Y,Unclear scenario:On-net to Off-net PGW, On-net to Off-net Partner IP:BSO,PGWO
BW-CDR-20130712084500-2-AC162DB137C8-037611-000.csv_rejected_4214495-20130718193659.log:0073014403AC162DB137C820130712064057.4190+020000,NORTON,Normal,+27873510673,,Originating,+27873510673,Public,0319041547,20130712064057.419,0+020000,Yes,20130712064102.933,20130712064106.378,016,VoIP,,0319041547,interlat,to,0319041547,,local,10.0.108.11:5060,BW084057422120713-197885586@196.35.130.5,G729/8000/1,10.0.4.117,B35406D4-E9F611E2-ABA8E293-3FE9594E@192.168.35.2,,,,NORTON-JUNE,,,,,,,,,,y,public,,834288043:143,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,0873510673@isvoip.net,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,3.445,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,Network,,,,,,,,,,,,,,,,+27873510673,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,|Y,Unclear scenario:On-net to Off-net PGW, On-net to Off-net Partner IP:BSO,PGWO
BW-CDR-20130712084500-2-AC162DB137C8-037611-000.csv_rejected_4214495-20130718193659.log:0073023480AC162DB137C820130712064304.9150+020000,RCS,Normal,+27215974210,,Originating,+27215974210,Anonymous,0153075587,20130712064304.915,0+020000,Yes,20130712064321.138,20130712064328.386,016,VoIP,,0153075587,interlat,to,0153075587,,local,10.0.108.11:5060,BW0843049181207131443732226@196.35.130.5,G729/8000/1,10.0.8.70,4C4196DA-E9F811E2-8D55E8C5-EFB54917@196.14.62.229,,,,RCS-CPT,,,,,,,,,,y,public,,834009203:4927,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,0215974210@isvoip.net,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,9.518,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,Network,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,|Y,Unclear scenario:On-net to Off-net PGW, On-net to Off-net Partner IP:BSO,PGWO
BW-CDR-20130712085000-2-AC162DB137C8-037612-000.csv_rejected_4214161-20130718193659.log:0073036207AC162DB137C820130712064336.1780+020000,VELOCITY,Normal,+27313278050,,Originating,+27313278050,Anonymous,0129918581,20130712064336.178,0+020000,Yes,20130712064509.255,20130712064643.114,016,VoIP,,0129918581,interlat,to,0129918581,,local,10.0.108.11:5060,BW0843361801207131128716458@196.35.130.5,G729/8000/1,10.0.12.18,298B8C35-EA0211E2-A4B9A37C-B09F2E77@196.35.99.82,,,,VELOCITY_UMH,,,,,,,,,,y,public,,834030275:820,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,0313278050@isvoip.net,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,185.546,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,Network,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,|Y,Unclear scenario:On-net to Off-net PGW, On-net to Off-net Partner IP:BSO,PGWO
BW-CDR-20130712085500-2-AC162DB137C8-037613-000.csv_rejected_4213301-20130718193659.log:0073051501AC162DB137C820130712064903.3170+020000,SAICOM-CORP,Normal,+27873516000,,Originating,+27873516000,Anonymous,0357870631,20130712064903.317,0+020000,Yes,20130712064925.379,20130712065037.397,016,VoIP,,0357870631,interlat,to,0357870631,,local,10.0.108.11:5060,BW084903320120713506949740@196.35.130.5,G729/8000/1,10.0.24.50,04D109C9-FEFB-4CCC-9282-8FBCB41BA855,,,,CORPORATE,,,,,,,,,,y,public,,834488621:4,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,0873516000@isvoip.net,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,72.018,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,Network,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,|Y,Unclear scenario:On-net to Off-net PGW, On-net to Off-net Partner IP:BSO,PGWO
BW-CDR-20130712085500-2-AC162DB137C8-037613-000.csv_rejected_4213301-20130718193659.log:0073065650AC162DB137C820130712065223.3030+020000,SEESA,Normal,+27219109256,,Originating,+27219109256,Public,0214214016,20130712065223.303,0+020000,Yes,20130712065340.993,20130712065404.457,016,VoIP,,0214214016,local,to,0214214016,,local,10.0.108.11:5060,BW0852233061207131522387561@196.35.130.5,G729/8000/1,10.0.8.97,89650630519337@192.168.3.20,,,,SEESA-CPT,,,,,,,,,,y,public,,834541151:0,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,0219109256@isvoip.net,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,98.267,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,Network,,,,,,,,,,,,,,,,+27219109256,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,|Y,Unclear scenario:On-net to Off-net PGW, On-net to Off-net Partner IP:BSO,PGWO
BW-CDR-20130712090500-2-AC162DB137C8-037615-000.csv_rejected_4216427-20130718195021.log:0073096717AC162DB137C820130712065912.3740+020000,AVUSA,Normal,+27114714072,,Originating,+27114714072,Anonymous,0110352300,20130712065912.374,0+020000,Yes,20130712065916.303,20130712070141.409,016,VoIP,,0110352300,private,to,0110352300,,local,10.0.108.11:5060,BW085912376120713378639972@196.35.130.5,G729/8000/1,10.0.24.157,34C7D79D-E9F811E2-9821C520-E97C0089@172.17.84.2,,,,AVUSA-INDUSTRIA,,,,,,,,,,y,public,,834620763:1,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,0114714072@isvoip.net,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,147.752,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,Network,,,,,,,,,,,,,,,,+27114714072,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,|Y,Unclear scenario:On-net to Off-net PGW, On-net to Off-net Partner IP:BSO,PGWO
BW-CDR-20130712090500-2-AC162DB137C8-037615-000.csv_rejected_4216669-20130718195021.log:0073106024AC162DB137C820130712070250.8060+020000,MURROB-VOICE,Normal,+27114561000,,Originating,+27114561000,Anonymous,0115932184,20130712070250.806,0+020000,Yes,20130712070252.011,20130712070356.550,016,VoIP,,0115932184,local,to,0115932184,,remote,10.0.108.11:5060,BW0902508091207131728834809@196.35.130.5,G729/8000/1,10.0.0.102,f398e80-1df1aa1a-234d8c-a1086e0a@10.110.8.161,,,,DRC,,,,,,,,,,y,public,,833824631:1598,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,0114561000@isvoip.net,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,64.538,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,Network,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,|Y,Unclear scenario:On-net to Off-net PGW, On-net to Off-net Partner IP:BSO,PGWO
BW-CDR-20130712091500-2-AC162DB137C8-037617-000.csv_rejected_4215097-20130718193659.log:0073140848AC162DB137C820130712065558.6470+020000,FUEL-RTT,Normal,+27115521219,,Originating,+27115521219,Public,0538321273,20130712065558.647,0+020000,Yes,20130712065607.980,20130712071215.187,016,VoIP,,0538321273,private,to,0538321273,,local,10.0.108.20:5060,BW085558650120713-2132867107@196.35.130.5,G729/8000/1,10.0.4.160,7a191f540c17c53359075e850163b863@10.98.20.4:5060,,,,KEMPTONPARK-DC,,,,,,,,,,y,public,,834586903:0,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,0115521219@isvoip.net,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,970.546,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,Network,,,,,,,,,,,,,,,,+27115521219,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,|Y,Unclear scenario:On-net to Off-net PGW, On-net to Off-net Partner IP:BSO,PGWO
-bash-3.2$ grep Originating BW*2013*|grep -v "ACCOUNT NOT FOUND"|grep -v "Unknown called number"|grep -v "Y,Unknown called on-net party"|grep -v "|Y,Unknown calling on-net party"|grep -v "Y,ACCOUNT IN RECALC."|grep "|Y,Unclear scenario:On-net to Off-net PGW, On-net to Off-net Partner IP:BSO,PGWO"|head
-bash-3.2$

Thanking you in advance.

2

Not clear.
Can u give us your input file and what output you expect?

3

So, could you put all the unwanted patterns in a file, similar to this:-

ACCOUNT NOT FOUND
Unknown called number
Y,Unknown called on-net party
Y,ACCOUNT IN RECALC.

If so, you can then:-

grep -vf exclude_patterns_file   data_file|grep "|Y,Unclear scenario:On-net to Off-net PGW, On-net to Off-net Partner IP:BSO,PGWO"

....but they seems awfully long search patterns.

Is this the right sort of approach? Can you reduce and neaten the patterns and not include/exclude the wrong things?

Eg. to have a pattern that looks for abc or abd, you can use something more like:-

grep "ab[cd]" file

Just as a matter of security, you're not publishing customer contact details here are you? If so, please sanitise them. I wouldn't like the data to be misused.

I hope that this helps, but if I've missed the point, please clarify what you actually need with some sanitised sample input from a single file and your expected output.

Thanks,

Robin
Liverpool/Blackburn
UK