How do you [e]grep for multiple values within multiple files?

Applications UNIX Desktop Questions & Answers
1

Hi
I'm sure there's a way to do this, but I ran out of caffeine/talent before getting the answer in a long winded alternate way (don't ask :wink: )

The task I was trying to do was scan a directory of files and show only files that contained 3 values:
I940
5433309
2181

I tried many variations of grep and egrep and the last one I tried before I gave up on a single line resolution was:

egrep -i  'I940|5433309|:2181' *

3000+ files of which only 2 contained all 3 values.
Is there a single line command to show the only files with all 3 matching criteria?

2

Try:

grep -l I940 * | xargs grep -l 5433309 | xargs grep -l 2181
3

Hi.

Here is a shell script that encapsulates an awk script. The awk reads each line of each file once to ensure that each file contains at least one match to the multiple strings.

This example uses 2 strings and looks through 4 files, but can be easily extended for more strings. One advantage is that each file is read only once. Another is that when all strings have matched at least once, the operation skips to the next file. For a small number of files, the awk probably is no faster than multiple greps, but for the grep, you can imagine that for 3000 files, if the first keyword appears in 2000 files, then those 2000 files will be read at least twice, and so on. On the other hand, awk will use more CPU time than grep. For simple alpha-numeric strings, fgrep might be slightly faster than plain grep.

Both this approach and the suggestion from bartus11 will require some work to fit a specific problem. Here is the awk:

#!/usr/bin/env bash

# @(#) s1	Demonstrate single-pass to find multiple strings in files.

pe() { for _i;do printf "%s" "$_i";done; printf "\n"; }
pl() { pe;pe "-----" ;pe "$*"; }
db() { ( printf " db, ";for _i;do printf "%s" "$_i";done;printf "\n" ) >&2 ; }
db() { : ; }
C=$HOME/bin/context && [ -f $C ] && $C awk

FILES="data*"
pl " Data files:"
for i in $FILES
do
  pe
  pe " Data file $i:"
  cat $i
done

pl " Results, files must contain \"a\" AND \"b\":"
# "c" is not used for testing, and can be eliminated.
awk '
BEGIN	{ a = b = c = 0 }
FNR == 1	{ a = b = c = 0 }
/a/	{ a++ }
/b/	{ b++ }
/c/	{ c++ }
a>0 && b>0	{ print FILENAME ; nextfile }
' $FILES

exit 0

producing:

% ./s1

Environment: LC_ALL = C, LANG = C
(Versions displayed with local utility "version")
OS, ker|rel, machine: Linux, 2.6.26-2-amd64, x86_64
Distribution        : Debian GNU/Linux 5.0.8 (lenny) 
bash GNU bash 3.2.39
awk GNU Awk 3.1.5

-----
 Data files:

 Data file data1:
a
b

 Data file data2:
b
a
c

 Data file data3:
c
a

 Data file data4:
a b
c
d

-----
 Results, files must contain "a" AND "b":
data1
data2
data4

See man pages for details.

Best wishes ... cheers, drl

4 
grep -i -e I940 -e 5433309 -e 2181 *
5

Thanks for your replies peeps

I'll try those to see how they get on.

CW
P.S. Thanks for the awesome reply DRL, but that's a tough one to work with when the enquirer is on the other end of the phone waiting for an answer :slight_smile:
Pretty cool tho :b: