Compare two files and output diff to third file

Shell Programming and Scripting
1

I have made several attempts to read two files of ip addresses and eliminate records from file1 that are in file2.

My latest attempt follows. Everything works except my file3 is exactly the same as file1 and it should not be.

# !/usr/bin/bash
#
# NoInterfaces
# Utility will create a file of ip addresses( file1) from an ARP file from network admin.
# Utility will create a file of interface ip addresses(file2) from an ARP file from network admin.
# Utility will create a file of user ip addresses(file3) from file1 and file2 created within utility
#

# set some variables
DIR=/export/home/gordonl
file1=$DIR/ipstrings     # Large file containing all arp ip strings
file2=$DIR/intfip        # small file containing interface ip strings
file3=$DIR/output.txt    # output file contains all of file1 - file2
# null the files
>$file1
>$file2
>$file3

# Locate all Internet IP Address strings from RtrARP
grep Internet $DIR/RtrARP | cut -d" " -f3 | sort -n -t . -k 1,1 -k 2,2 -k 3,3 -k 4,4 > $file1

# Locate all Router Interface strings
grep "ip address 1" $DIR/RtrARP | cut -d" " -f4 | sort -n -t . -k 1,1 -k 2,2 -k 3,3 -k 4,4 > $file2

while read HOST
do
   grep -v "$HOST" $file1 > $file3
done < $file2

thanks in advance for any help.

JB

2

Replace:

while read HOST
do
   grep -v "$HOST" $file1 > $file3
done < $file2

with:

grep -v -f $file2 $file1 > $file3

Regards

3

You can search the forum or just look here at the bottom of this page for similar problems.
If you can't find anything ? try to post some samples of your file.

4

Thanks Franklin.... just what I needed. I had to use another version of grep to allow the -f to work.

/usr/xpg4/bin/grep -v -f $file2 $file1 > $file3

danmero,

I did search and came up with lots of hits, and I tried several of them, using comm, uniq, diff, etc. I felt I needed to show my utility to the group and someone would see my simple processes and help me keep it that way. I would love to be able to just sit down and sream out some perl code, but I am a few lessons short of that at this time.

Thanks again to all of this board. You have helped me every time I came to you.

JB

5

You want the output of records that are unique to file1.
assuming both files have a record layout of
xxx.xxx.xxx.xxx ie., a single column

awk 'FILENAME=="file1" { if( $0 in arr) {continue} else {print $0 } }
      FILENAME=="file2" {arr[$0]++ }'  file2 file1
6

Jim,

I tried that one before posting, but could never get it to give me the expected output. Maybe I did replace the variables in your code correctly. Also all three positions may not be fillled in the last three octets, so it may be xxx.xx.xxx.x or most any way we folks write ip addresses.

Franklin,

I spoke too soon. My output using grep eliminated not just xxx.xxx.xxx.1 but anything with a 1 in the first position of the last octet.

JB

7

What's the format of the data in the files? Perhaps you want fgrep and/or grep -x and/or massage the data in file1 into proper regular expressions.

8

Thank you era. That keen eye from herding cats has found the option for me.

The ip strings can be variable length within the octets as 111.111.111.111 or 111.11.1.11 and the -x option allowed me to find an exact match on the whole line.

I was using

/usr/xpg4/bin/grep -v -f $file2 $file1 > $file3

on a Solaris 10 box, which is thought by me to be the same as fgrep, but my reference did not show the -x option under either grep of fgrep, but I looked closer and it told me the options for fgrep, egrep and some versions of grep were the same. I looked in the egrep def and sure enough, my reference paraphrases era.

This

/usr/xpg4/bin/grep -vxF -f $file2 $file1 > $file3

and this one

fgrep -xv -f "$file2" $file1 > $file3

worked correctly.

Thanks again to the group for your help.

JB

9

I was just browsing through.. nice solution!! :b: :smiley: