fgrep command

morbid_angel · December 23, 2009, 2:15am

How can we use fgrep command to search pattern on perticular field on a file.
eg : I have a parren file having format
cat patternfile
SPA16S199982
SPA5S26330

I want to seach these pattern on FIRST field of File2(since other field has
same data) and that too maching entirely.

The command that i am using is fgrep -f -x patternfile File2. How to modify this command for grepping on first column.

Thanxx in advance

clx · December 23, 2009, 2:42am

if using fgrep is not a compulsion:

while read LINE;do grep -w "^$LINE" File2;done < patternfile

or with awk this should also work

awk -F"|" 'BEGIN{while(getline<"patternfile") arr[$1]=1 } ; arr[$1] ==1 {print $0 } ' File2

Scrutinizer · December 23, 2009, 2:49am

You can not use fgrep for this purpose, you have to use grep. Also you can not use the -x option since that matches the whole line, plus you can not use the pattern unmodified. So you either have to modify the pattern file or modify the patterns on the fly:

grep -wf <(sed s'/^/^/' patternfile) File2

or use awk:

awk -F '|' 'NR==FNR{A[$1]=1;next}A[$1]' patternfile File2

thegeek · December 23, 2009, 2:49am

did you mean word match ?

13:17:45 : tmp :cat pat
SPA16S199982
SPA5S26330
13:17:46 : tmp :fgrep -w -f pat example
SPA16S199982P|Diamond Necklace|SPA16S199982|1
SPA16S199982|Diamond Necklace|SPA16S199982|1
SPA5S26330|Diamond Necklace|SPA16S199982|1

Or if you wanted only the first field to match, then give ^ at the pattern as

$ cat pat
^SPA16S199982
^SPA5S26330

$ grep -w -f pat example
SPA16S199982|Diamond Necklace|SPA16S199982|1
SPA5S26330|Diamond Necklace|SPA16S199982|1

morbid_angel · December 23, 2009, 3:14am

Hi All, the awk command is working fine Thanks a lot .Now the major think is my pattern file and main file has 5million data. In that case will the awk command be faster ?

Scrutinizer · December 23, 2009, 3:30am

If the awk command can handle the 5M records I think it will be plenty fast.

yazi0127 · December 23, 2009, 3:52am

#!/bin/sh

PATTERNS=`cat patternfile`

for pat in $PATTERNS
do
    cat File2 | grep "^$pat\>"
done

morbid_angel · December 23, 2009, 5:33am

All the Commands are working fine.But the problem is , the commands are taking huge time when the files have records around 5M

Scrutinizer · December 23, 2009, 5:36am

How long does it take exactly?

morbid_angel · December 23, 2009, 8:17am

#!/bin/sh

PATTERNS=`cat patternfile`

for pat in $PATTERNS
do
cat File2 | grep "^$pat\>"
done

this command is fine . but the problem is this is very slow if have millions records

---------- Post updated at 08:17 AM ---------- Previous update was at 08:11 AM ----------

the above command is very slow if we have millions of records in the file

cfajohnson · December 23, 2009, 11:47am

morbid_angel:

#!/bin/sh

PATTERNS=`cat patternfile`

for pat in $PATTERNS
do
cat File2 | grep "^$pat\>"
done
this command is fine . but the problem is this is very slow if have millions records

---------- Post updated at 08:17 AM ---------- Previous update was at 08:11 AM ----------

the above command is very slow if we have millions of records in the file

Why are you using cat?