Compare an item in one line of a file against an item in the next line of the same file

Hi all,

I found the following old thread very useful and in particular the very helpful post by @frans

while read A # read line and put in variable A. If unreadable (EOF) the break the loop.
do
    read B # read next line and put in variable B
    ((i+=2)) # increment variable i by 2 (like i=i+2)
    # [ "$A" != "$B" ] && echo "Lines $((i-1)) and $i are different"
    # can be written
    if [ "$A" != "$B" ] # if values of variables A and B are different ('[ ]' stands for 'test')
    then echo "Lines $((i-1)) and $i are different" # OK ?
    fi
done < inputfile # tells the loop to read from inputfile (else, it would read from keyboard.)

However, I am looking to try and compare one item / word from the first line to see if that same item is part of the second line

So in a sense, I'm trying to say something like
if $A | grep "XXX" = $B | grep "XXX" then do ..........

I know the position of the word in the first line so I tried using if [ "$A" | awk {'print $5'} = "$B" | awk {'print $2'} ]
but no joy. It didn't like the syntax of that.

could you provide a sample/representative snapshot of your input file inputfile and a desired output.

Not clear. Do you want to compare every other line to its predecessor? So, operate on pairs of lines? Or, on every single line? And compare predecessor's field 5 to actual line's field 2? Is it possible that read B hits EOF ? How do you handle that?

Thanks for taking the time to reply

I am trying to work with a firewall ruleset. Every DROP rule should have a partner LOG rule. But from the following, you can see that two rules do not have a corresponding LOG rule. So I am trying to process the file and identify all rules I have a DROP but no LOG. The two offenders in this selection below are in BOLD

set security firewall name Private-to-AA rule 25 action 'drop'
set security firewall name Private-to-AA rule 9999 action 'drop'
set security firewall name Private-to-AA rule 9999 'log'
set security firewall name Private-to-BB default-action 'drop'
set security firewall name Private-to-BB 'default-log'
set security firewall name Private-to-CC default-action 'drop'
set security firewall name Private-to-CC 'default-log'
set security firewall name Private-to-CC rule 9999 action 'drop'
set security firewall name Private-to-CC rule 9999 'log'
set security firewall name Private-to-DD default-action 'drop'
set security firewall name Private-to-DD 'default-log'
set security firewall name Private-to-DD rule 9999 action 'drop'
set security firewall name Private-to-DD rule 9999 'log'
set security firewall name local default-action 'drop'
set security firewall name local rule 10 action 'drop'
set security firewall name local rule 10 'log'
1 Like

a bit verbose (assuming 'drop' comes first), but....
awk -f cham.awk ruleSetFile.txt where cham.awk is:

{
  idx=""
  slide=(/drop/)?2:1
  for(i=1;i<=NF-slide;i++)
     idx=(i==1)?$i:idx $i
}
/drop/{d[idx]=$0;next}
/log/ && (idx in d) {delete d[idx]}
END {
  for(i in d)
    print d
}

results in:

set security firewall name Private-to-AA rule 25 action 'drop'
set security firewall name local default-action 'drop'

Great thanks, I'll give that a go. I'm a real BASH noob, so I'd never have managed to get that solution, or anything close to it I think.

Do I need to make a separate script with the contents bring the code in the code section, and then call it from the awk command ? Or can I put the whole lot into a single command that I can paste into a command prompt ?

for simplicity sake.... create a file cham.awk with the content as indicated in the post and execute the command line as indicated in the post.
There're other ways to execute what's been provided, but for now let's stick with the provided paradigm.

Many thanks @vgersh99

I deeply appreciate your assistance with this code. I have to run this command on over 50 devices, so I would really much prefer to run the command straight as creating a secondary file on all those devices will create a lot more work.

I don't follow... How creating a single script file will "create a lot more work" whether it gets run against one file/device or 50?
But if you insist:

#!/bin/ksh
awk '
{
  idx=""
  slide=(/drop/)?2:1
  for(i=1;i<=NF-slide;i++)
     idx=(i==1)?$i:idx $i
}
/drop/{d[idx]=$0;next}
/log/ && (idx in d) {delete d[idx]}
END {
  for(i in d)
    print d
}' ruleSetFile.txt

if you have multiple rule files (1 per device), you can use the above and change it to iterate over your rule fileS.

3 Likes

You're an absolute star - thank you so much.

My issue was that I have to run it (exactly the same command request) on 50+ (nearer to 100) devices. So being able to logon to each device and just paste in a script on the commd line is so much simpler that needing to create a file on each device.

Are you saying you're logging in to each "device" and running the script manually?
You think this would scale?
Why not to invest time/effort into automating this repetitive task?!