Need a script or one-liner to purge lines from a file.

i all.

This one sounds so simple, but I can't get it to work. I need to delete lines with certain keywords from a file.

I have a file called defaultRules, with keywords:

IPSEC_AH
IKE_UDP
IPMP_TEST_IFACE2

Then, I have another file called rules.txt with some rules:

'344.','IPSEC_ESP','','','','','1.','0.','0.','0.','0.','5.','1.','Allow IPSec ESP Packets on all IP interfaces'
'345.','IPSEC_AH','','','','','1.','0.','0.','0.','0.','6.','1.','Allow IPSec AH Packets on all IP interfaces'
'346.','IKE_TCP','','','','','1.','500.','0.','0.','0.','3.','1.','Allow IKE Negotiation on all IP interfaces'
'347.','IKE_UDP','','','','','1.','500.','0.','0.','0.','2.','1.','Allow IKE Negotiation on all IP interfaces'
'348.','CLUSTER_IC_NODE1','clusternode1-priv','','','','1.','0.','0.','0.','0.','4.','1.','Allow cluster inter-connect packets from node 1'
'349.','CLUSTER_IC_NODE2','clusternode2-priv','','','','1.','0.','0.','0.','0.','4.','1.','Allow cluster inter-connect packets from node 2'
'350.','ICMP_NODE_IP','ipmp_node_alias','','','','1.','0.','0.','0.','0.','1.','1.','Allow ICMP to Node IP'
'351.','ICMP_BMS','ipmp_bms','','','','1.','0.','0.','0.','0.','1.','1.','Allow ICMP to BMS IP'
'352.','IPMP_TEST_IFACE1','ipmp_test_alias_1','','','','1.','0.','0.','0.','0.','1.','1.','Allow IPMP Test Packets on 1st test interface'
'353.','IPMP_TEST_IFACE2','ipmp_test_alias_2','','','','1.','0.','0.','0.','0.','1.','1.','Allow IPMP Test Packets on 2nd test interface'

The objective is to remove all lines in rules.txt that contain the keywords in defaultRules, so I tried this by first renaming the keywords to 'unused' and then removing all lines that contain 'unused':

for a in `cat defaultRules` ; do sed -e 's/$a/unused/' rules.txt | grep -v unused > rules.out ; done

It did nothing, rules.txt and rules.out are identical.

Any ideas?

Thanks in advance,
BRH

grep -vf defaultRules rules.txt

I've only used Perl for scripting, so I'll give you some advice for Perl.

Read rules.txt and read each line.
If the line has any of those keywords in defaultRules, then do nothing.
Else, put that line into an output file rules.out

open(INPUTFILE, "<rules.txt");
open(OUTPUTFILE, ">rules.out");

while (<INPUTFILE>) {
    my $line = $_;
    if ($line =~ m/IPSEC_AH/i || $line =~ m/IKE_UDP/i || $line =~ m/IPMP_TEST_IFACE2/i) {
             print OUTPUTFILE "$line";
    }
}

close(INPUTFILE);
close(OUTPUTFILE);

Hi danmero, which grep are you using, I'm on a Solaris 10 system:

/usr/bin/grep -vf defaultRules rules.txt
/usr/bin/grep: illegal option -- f
Usage: grep -hblcnsviw pattern file . . .

Regards,
BRH

---------- Post updated at 03:58 PM ---------- Previous update was at 03:56 PM ----------

Thanks for the reply, kooshi, however, this won't workin my case. The full defaultRules is over 100 lines.

Regards,
BRH

Try /usr/xpg4/bin/grep on Solaris ?!

HI dan, no luck with this one either, it didn't complain, but it did nothing.

Regards,
BRH

is there one in /usr/ucb/grep ? or give a try with egrep ?

A bit convoluted but it works on my system:

tr '\n' '|' < rules.dat | sed 's/|$//' | xargs -iRULE egrep -v RULE data.dat

1) Change newlines to pipes
2) Remove trailing pipe
3) Pass concatenated rule as expression to grep -v

awk 'FNR==NR{a[$0]; next} {for (i in a) if (index($0,i)) next; print}' defaultRules rules.txt

Regards,
Alister

Hmm, do you have /usr/xpg4/bin/grep ?
Base on /usr/xpg4/bin/grep manual the solution should work as expected

Thanks for your reply, but nope and nope.

Regards,
BRH

---------- Post updated at 05:05 PM ---------- Previous update was at 04:52 PM ----------

Hi Dan.
My mistake, /usr/xpg4/bin/grep did work. I changed user and hit the wrong directory.

Regards,
BRH

---------- Post updated at 05:06 PM ---------- Previous update was at 05:05 PM ----------

Thanks for everyone who replied, much appreciated.

Regards,
BRH