How to iterate Grep via all patterns provided in an input file?

UNIX for Beginners Q & A
1

When I use the following grep command with options -F and -f, its just displaying the text related to only the last pattern.

Command: $ grep -f pattern_file.txt input_file.txt

Output: doc-C2-16354

Even the following command yields the same output:
Command: $ grep -Ff pattern_file.txt input_file.txt

Contents of the pattern_file.txt:

A1
B3
C2

Contents of the input_file.txt:

doc-A1-151
file-A2-15646
table-A3-1654
file-B1-15654
doc-B2-15654
table-B3-13546
file-C1-164654
doc-C2-16354
table-C3-13565

Expected output:

doc-A1-151
table-B3-13546
 doc-C2-16354
2

Hello nsai,

Could you please try following and let me know if this helps you.

awk 'FNR==NR{a[$0];next} ($2 in a)' pattern_file  FS="-" input_file.txt

Thanks,
R. Singh

3

Hello RavinderSingh,
I'm still getting the same output "doc-C2-16354"
I executed the below command:

$ awk 'FNR==NR{a[$0];next} ($2 in a)' pattern_file.txt  FS="-" input_file.txt

Output: doc-C2-16354

Regards,
Sai

4

Hello Sai,

You could hit THANKS button for any helpful post for anyone. Coming to your question, I am getting perfect output as follows with my command see this:

awk 'FNR==NR{a[$0];next} ($2 in a)' pattern_file  FS="-" input_file.txt
doc-A1-151
table-B3-13546
doc-C2-16354

Could you please do cat -v Input_file where Input_file is your files and see if you have control M characters in them in case you have them then remove them by doing tr -d '\r < Input_file > temp_file && mv temp_file Input_file and let me know then how it goes?

Thanks,
R. Singh

1 Like
5

Yes I think at least the pattern_file is DOS format, where each search pattern have a \r at the end, but the last one, that is an incomplete line (that is completed by some versions of grep and awk; then the only the last pattern works).
Workaround: strip the trailing \r when reading from the pattern file.

awk 'FNR==NR {sub(/\r$/,""); a[$0]; next} ($2 in a)' pattern_file.txt FS="-" input_file.txt
6

Thanks so much, RavinderSingh. Thanks for all the information. I was unaware of the Ctrl M characters earlier.
I cleaned up the ^M characters from both the pattern_file.txt and input_file.txt. Then only the output showed up.

After the cleanup in both the files, even the command $ grep -Ff "pattern_file.txt" "input_file.txt" is now giving me the expected output.

Once again, Thanks a lot for providing the valuable information on the ^M characters and the AWK & cat commands for getting my issue resolved so quickly.

Regards,
Sai

8

Thank you, MadeInGermany. The command provided by you worked, without me meddling with the files.

Once again Thanks to you both (RavinderSingh13 and MadeInGermany) for helping me with the required commands.

------ Post updated at 09:10 AM ------

RavinderSingh,
Thanks for letting me know. I just hit the Thanks button as you mentioned. I'm new here on the forum, so in the process of getting myself familiarized with the available controls on this website.