Take a list if strings from a file and search them in a list of files and report them

Shell Programming and Scripting
1

I have a file 1.txt with the below contents.
-----cat 1.txt-----

1234
5678
1256
1234
1247

-------------------
I have 3 more files in a folder
-----ls -lrt-------

A1.txt
A2.txt
A3.txt

-------------------
The contents of those three files are similar format with different data values (All the three files are tab delimited)
-----cat A1.txt----

A   X   1234    B   1234
A   X   5678    B   1234
A   X   1256    B   1256

-------------------
-----cat A2.txt----

A   Y   8888    B   1234
A   Y   9999    B   1256
A   X   1234    B   1256

-------------------
-----cat A3.txt----

A   Y   6798    C   1256

-------------------
My objective is to do a search on all the A1,A2 and A3 (Only for the 3rd column of the TAB delimited file)for text in 1.txt
and the output must be redirected to the file matches.txt as given below.

/home/A1.txt:A   X   1234    B   1234
/home/A1.txt:A   X   5678    B   1234
/home/A1.txt:A   X   1256    B   1256
/home/A2.txt:A   X   1234    B   1256

Could someone please help with this?

2

Try:

awk 'NR==FNR{a[$0]=1}$3 in a{print FILENAME":"$0}' 1.txt A*
1 Like
3

Could you please explain how this works? Also I need to redirect the output to a file.

4

It is storing the contents of 1.txt into array "a", then it is checking if third column on the other files is present in that array. If so, it is printing the filename along with the line that the match occurred. To redirect the output use:

awk 'NR==FNR{a[$0]=1}$3 in a{print FILENAME":"$0}' 1.txt A* > matches.txt
1 Like
5

The same example works fine:b:. But when I apply the same to a real time scenario. It is not working.:confused:.

awk 'NR==FNR{a[$0]=1}$19 in a{print FILENAME":"$0}' 2.txt $RESPONSE_DIR/DOWNLOAD*.dat > 2matches.txt

I try to get a list of files from RESPONSE_DIR, the position is 19.
19th position is the last position in the file. Also there may be spaces with in the values ...

---------- Post updated at 11:56 AM ---------- Previous update was at 07:26 AM ----------

Also this is not working for more than 200 records. :eek:

6

Files are very similar i.e both are in Linux mode. Urgent situation. Please h

7

quite simple: use the "grep" utility with the "-f" (file) option. See "man grep" for a detailed explanation:

grep -f /file/with/key/values /path/to/A[123].txt > match.txt

I hope this helps.

bakunin

8

But this doesn't check the 19th field!!

9

I'm trying to achieve similar thing but in diff way :wall:

http://www.unix.com/shell-programming-scripting/169062-file-name-list-files.html\#post302564207

That is, I've a string to grep from a series of files, when it finds the string, it should redirect the srting+from which file it got that string! :rolleyes:

I'd tried using the above posted command, however I'm getting the below

> bash -x pleasework.sh
+ logpath=/opt/app/p3wtg1z2/sbpprod1/FW/home/UnifiedLogging
++ date +%Y%m%d
+ awk 'NR==FNR{a[$0]=1}$1 in a{print FILENAME":"$0}' 2.txt /opt/app/p3wtg1z2/sbpprod1/FW/home/UnifiedLogging/UL_20111013_Scheduler0.log.csv /opt/app/p3wtg1z2/sbpprod1/FW/home/UnifiedLogging/UL_20111013_Scheduler1.log.csv /opt/app/p3wtg1z2/sbpprod1/FW/home/UnifiedLogging/UL_20111013_Scheduler10.log.csv /opt/app/p3wtg1z2/sbpprod1/FW/home/UnifiedLogging/UL_20111013_Scheduler11.log.csv /opt/app/p3wtg1z2/sbpprod1/FW/home/UnifiedLogging/UL_20111013_Scheduler12.log.csv /opt/app/p3wtg1z2/sbpprod1/FW/home/UnifiedLogging/UL_20111013_Scheduler13.log.csv /opt/app/p3wtg1z2/sbpprod1/FW/home/UnifiedLogging/UL_20111013_Scheduler14.log.csv /opt/app/p3wtg1z2/sbpprod1/FW/home/UnifiedLogging/UL_20111013_Scheduler15.log.csv /opt/app/p3wtg1z2/sbpprod1/FW/home/UnifiedLogging/UL_20111013_Scheduler16.log.csv /opt/app/p3wtg1z2/sbpprod1/FW/home/UnifiedLogging/UL_20111013_Scheduler17.log.csv /opt/app/p3wtg1z2/sbpprod1/FW/home/UnifiedLogging/UL_20111013_Scheduler18.log.csv /opt/app/p3wtg1z2/sbpprod1/FW/home/UnifiedLogging/UL_20111013_Scheduler19.log.csv /opt/app/p3wtg1z2/sbpprod1/FW/home/UnifiedLogging/UL_20111013_Scheduler2.log.csv /opt/app/p3wtg1z2/sbpprod1/FW/home/UnifiedLogging/UL_20111013_Scheduler20.log.csv /opt/app/p3wtg1z2/sbpprod1/FW/home/UnifiedLogging/UL_20111013_Scheduler21.log.csv /opt/app/p3wtg1z2/sbpprod1/FW/home/UnifiedLogging/UL_20111013_Scheduler22.log.csv /opt/app/p3wtg1z2/sbpprod1/FW/home/UnifiedLogging/UL_20111013_Scheduler23.log.csv /opt/app/p3wtg1z2/sbpprod1/FW/home/UnifiedLogging/UL_20111013_Scheduler24.log.csv /opt/app/p3wtg1z2/sbpprod1/FW/home/UnifiedLogging/UL_20111013_Scheduler25.log.csv /opt/app/p3wtg1z2/sbpprod1/FW/home/UnifiedLogging/UL_20111013_Scheduler26.log.csv /opt/app/p3wtg1z2/sbpprod1/FW/home/UnifiedLogging/UL_20111013_Scheduler27.log.csv /opt/app/p3wtg1z2/sbpprod1/FW/home/UnifiedLogging/UL_20111013_Scheduler28.log.csv /opt/app/p3wtg1z2/sbpprod1/FW/home/UnifiedLogging/UL_20111013_Scheduler29.log.csv /opt/app/p3wtg1z2/sbpprod1/FW/home/UnifiedLogging/UL_20111013_Scheduler3.log.csv /opt/app/p3wtg1z2/sbpprod1/FW/home/UnifiedLogging/UL_20111013_Scheduler30.log.csv /opt/app/p3wtg1z2/sbpprod1/FW/home/UnifiedLogging/UL_20111013_Scheduler31.log.csv /opt/app/p3wtg1z2/sbpprod1/FW/home/UnifiedLogging/UL_20111013_Scheduler32.log.csv /opt/app/p3wtg1z2/sbpprod1/FW/home/UnifiedLogging/UL_20111013_Scheduler33.log.csv /opt/app/p3wtg1z2/sbpprod1/FW/home/UnifiedLogging/UL_20111013_Scheduler34.log.csv /opt/app/p3wtg1z2/sbpprod1/FW/home/UnifiedLogging/UL_20111013_Scheduler35.log.csv /opt/app/p3wtg1z2/sbpprod1/FW/home/UnifiedLogging/UL_20111013_Scheduler36.log.csv /opt/app/p3wtg1z2/sbpprod1/FW/home/UnifiedLogging/UL_20111013_Scheduler37.log.csv /opt/app/p3wtg1z2/sbpprod1/FW/home/UnifiedLogging/UL_20111013_Scheduler38.log.csv /opt/app/p3wtg1z2/sbpprod1/FW/home/UnifiedLogging/UL_20111013_Scheduler39.log.csv /opt/app/p3wtg1z2/sbpprod1/FW/home/UnifiedLogging/UL_20111013_Scheduler4.log.csv /opt/app/p3wtg1z2/sbpprod1/FW/home/UnifiedLogging/UL_20111013_Scheduler40.log.csv /opt/app/p3wtg1z2/sbpprod1/FW/home/UnifiedLogging/UL_20111013_Scheduler41.log.csv /opt/app/p3wtg1z2/sbpprod1/FW/home/UnifiedLogging/UL_20111013_Scheduler42.log.csv /opt/app/p3wtg1z2/sbpprod1/FW/home/UnifiedLogging/UL_20111013_Scheduler43.log.csv /opt/app/p3wtg1z2/sbpprod1/FW/home/UnifiedLogging/UL_20111013_Scheduler44.log.csv /opt/app/p3wtg1z2/sbpprod1/FW/home/UnifiedLogging/UL_20111013_Scheduler45.log.csv /opt/app/p3wtg1z2/sbpprod1/FW/home/UnifiedLogging/UL_20111013_Scheduler46.log.csv /opt/app/p3wtg1z2/sbpprod1/FW/home/UnifiedLogging/UL_20111013_Scheduler47.log.csv /opt/app/p3wtg1z2/sbpprod1/FW/home/UnifiedLogging/UL_20111013_Scheduler5.log.csv /opt/app/p3wtg1z2/sbpprod1/FW/home/UnifiedLogging/UL_20111013_Scheduler6.log.csv /opt/app/p3wtg1z2/sbpprod1/FW/home/UnifiedLogging/UL_20111013_Scheduler7.log.csv /opt/app/p3wtg1z2/sbpprod1/FW/home/UnifiedLogging/UL_20111013_Scheduler8.log.csv /opt/app/p3wtg1z2/sbpprod1/FW/home/UnifiedLogging/UL_20111013_Scheduler9.log.csv
awk: syntax error near line 1
awk: bailing out near line 1

Where 'a' is ORA (the search string!!)

Thank you in advance :cool: