Grep -w ip address from a file isnt working need help

Shell Programming and Scripting
1

I have a file "file1" that contains several ip address [only ip address that i want to search in file2], and the "file2" contains several records [close to 100000], each line in file2 contains somewhere the ip address that i am searching in the file1

I use the unix command grep -w 

for i in `cat file1`
do
grep -w "$i" file2 >> file3
done

Sample Content of "file1" is shown below:

111.44.232.243
111.33.2.3


Sample Content of file2
SWITCHFEED|~|VENDORNAME=CISCO|~|VENDORMODEL=CISCO2192|~|SWITCH=XYZ1|~|IPADDRESS=111.44.232.243|~|
SWITCHFEED|~|VENDORNAME=CISCO|~|VENDORMODEL=CISCO2192|~|SWITCH=XYZ1|~|IPADDRESS=111.44.124.111|~|
SWITCHFEED|~|VENDORNAME=CISCO|~|VENDORMODEL=CISCO2192|~|SWITCH=XYZ1|~|IPADDRESS=111.33.213.222|~|
SWITCHFEED|~|VENDORNAME=CISCO|~|VENDORMODEL=CISCO2192|~|SWITCH=XYZ1|~|IPADDRESS=111.33.213.129|~|
SWITCHFEED|~|VENDORNAME=CISCO|~|VENDORMODEL=CISCO2192|~|SWITCH=XYZ1|~|IPADDRESS=111.33.213.204|~|
SWITCHFEED|~|VENDORNAME=CISCO|~|VENDORMODEL=CISCO2192|~|SWITCH=XYZ1|~|IPADDRESS=111.33.213.180|~|
SWITCHFEED|~|VENDORNAME=CISCO|~|VENDORMODEL=CISCO2192|~|SWITCH=XYZ1|~|IPADDRESS=111.33.213.8|~|
SWITCHFEED|~|VENDORNAME=CISCO|~|VENDORMODEL=CISCO2192|~|SWITCH=XYZ1|~|IPADDRESS=111.33.213.232|~|
SWITCHFEED|~|VENDORNAME=CISCO|~|VENDORMODEL=CISCO2192|~|SWITCH=XYZ1|~|IPADDRESS=111.33.213.233|~|
SWITCHFEED|~|VENDORNAME=CISCO|~|VENDORMODEL=CISCO2192|~|SWITCH=XYZ1|~|IPADDRESS=111.33.213.244|~|
SWITCHFEED|~|VENDORNAME=CISCO|~|VENDORMODEL=CISCO2192|~|SWITCH=XYZ1|~|IPADDRESS=111.33.233.17|~|
SWITCHFEED|~|VENDORNAME=CISCO|~|VENDORMODEL=CISCO2192|~|SWITCH=XYZ1|~|IPADDRESS=111.33.2.3|~|

the desired output in file3 should be the first record from file2 and last record from file2 but because the second ip in file1 is "111.33.2.3" is greps everything from file2 as it treats dots in [.2.3]

Any help in this regard is much appreciated.

2

grep -F finds literal strings - it turns off regular expressions (the problem you have with the dot character).

3 
user1@server2% grep -h
Usage: grep -hblcnsviw pattern file . . .

I dont see the Option grep -F for the grep, I am on SunOS 5.10

---------- Post updated at 06:13 PM ---------- Previous update was at 06:08 PM ----------

/usr/xpg4/bin/grep
/usr/bin/grep
user1@server2% /usr/xpg4/bin/grep -F
Usage:  grep [-c|-l|-q] [-bhinsvwx] pattern_list [file ...]
        grep [-c|-l|-q] [-bhinsvwx] [-e pattern_list]... [-f pattern_file]... [file...]
        grep -E [-c|-l|-q] [-bhinsvx] pattern_list [file ...]
        grep -E [-c|-l|-q] [-bhinsvx] [-e pattern_list]... [-f pattern_file]... [file...]
        grep -F [-c|-l|-q] [-bhinsvx] pattern_list [file ...]
        grep -F [-c|-l|-q] [-bhinsvx] [-e pattern_list]... [-f pattern_file]... [file...]

user1@server2% /usr/bin/grep -h
Usage: grep -hblcnsviw pattern file . . .
4

Would any of these suggestions work?

/usr/xpg4/bin/grep -Ff file1 file2 | /usr/xpg4/bin/awk -F"[=|]" '{print $13}' >> file3

or

/usr/xpg4/bin/awk 'FNR==NR {s[$1]; next} $13 in s {print $13}' file1 FS="[=|]" file2 >> file3

If you want the whole line where the ip is found in file2, then

/usr/xpg4/bin/grep -Ff file1 file2 > file4

or

/usr/xpg4/bin/awk 'FNR==NR {s[$1]; next} $13 in s' file1 FS="[=|]" file2 > file4
5

I do need the whole line where the ip is found in file2,

This solution did work for the examples I provided in this case but

/usr/xpg4/bin/grep -Ff file1 file2 > file4

when I apply the same solution when the file1 has more than 5000 ips that i am trying to search in file2 [which happens to have more than 100000 records ]and it fails,

6

Hello knijjar,

Please always do mention your O.S details from very first post itself, also try to provide sample Input_file and expected results with all the conditions you have. Could you please try following and let me know if this helps you.

awk -F'[|=]' 'FNR==NR{a[$0];next} ($13 in a)' Input_file1   Input_file2

On a Solaris/SunOS system, change awk to /usr/xpg4/bin/awk , /usr/xpg6/bin/awk , or nawk .

Thanks,
R. Singh

7

awk -F'[|=]' 'FNR==NR{a[$0];next} ($13 in a)' Input_file1 Input_file2

Which parameter is in $13 from Input_file2

8

You can find out, yourself, with a little test:

echo "SWITCHFEED|~|VENDORNAME=CISCO|~|VENDORMODEL=CISCO2192|~|SWITCH=XYZ1|~|IPADDRESS=111.44.232.243|~|" | /usr/xpg4/bin/awk -F"[=|]" '{print $13}'

Output:

111.44.232.243

or

/usr/xpg4/bin/awk -F"[=|]" '{print $13}' file2
9

Hello knijjar,

$13 represents the IP address column of Input_file2, I am adding complete explanation of code here, please do let me know if you have any queries on same.

awk -F'[|=]'   '     ##making field separator as |(pipe) and = here.
FNR==NR{             ##FNR==NR condition will be TRUE only when first Input_file named Input_file1 is being read, FNR and NR both represents the number of lines in an Input_file the only difference is FNRs value gets RESET once awk starts reading next Input_file and NRs value will be keep increasing till all the Input_file(s) are read.
a[$0];               ##creating an array named a whose index is current line of Input_file1.
next                 ##next is awk in-built keyword which skips all further statements.
}
($13 in a)           ##This condition will be running only when second Input_file named Input_file2 is being read, checking here if $13(field 13) which has IP address of Input_file2 is present in array a or NOT if present then no action is mentioned, so default action print of current line of Input_file2 will happen here.
' FiLE1 FiLE2        ##Mentioning names of Input_file(s), Input_file1 and Input_file2 here.

Thanks,
R. Singh

10

Thanks RavindeSingh13 / Aia, for the solution you guys provided, it has worked, this is exactly what I was looking for. Made my life so easy !!! FYI, I will remember to post the OS and the version upfront going forward