If contents of A are in B then move the common contents to C

Shell Programming and Scripting
1

Hallo Team,

I have 2 .csv files file A has 47600 lines and file B has 67000 lines

FILEA

SD0o9rb01-1d320ddbcc8d220f572739ebed5f58d1-v300g00
SD8bt0101-a0810bfe0e3396060126ec51b30dac0a-v300g00
SD05sce01-cb056af347ed4651f29eb3c3e9addbd6-v300g00
SDp9vv101-4179e27012ee1a250339025b6e7dd628-v300g00
SDud4hd01-ecbf99d937c192b48c96f5daca515769-v300g00
SD2jjk601-ac387dbd7f651c35c772e927d5f73a5f-v300g00
SD7k6g801-50114e1d12e0a1065fd8e26c8e2b7fd3-v300g00
SD5j0fc01-bc6daaf345754791d93acf70f19ec7b8-v300g00
SDal08c01-1c7d08b47eac029b590e3575cb66c661-v300g00
SD23ova01-a3861f173d70f18b0d955a8d9cc8e812-v300g00
SD4lmjf01-8eeeecccb17b4137ec3332cc115dc8d0-v300g00

FILEB

"SD0op6201-1430204309fffd58c09b1f76a0a6dc25-v300g00","3/1/2015","09:07:24","Unavailable","0873546275","087","5","0.0175","0.21"
"SDpfja101-b65120fdb4c27a155c203efb6126c0e1-v300g00","3/1/2015","13:55:39","0114358297","0873546275","087","5","0.0175","0.21"
"SDgnf6301-0f28db45ad48c4aab02a8dde712cc8f8-v300g00","3/1/2015","13:11:02","0114358297","0873546275","087","5","0.0175","0.21"
"SDifra101-344addc6f19f978b7155d06a0367542c-v300g00","3/1/2015","10:19:41","Unavailable","0873546275","087","14","0.049","0.21"
"SDg7o2201-be65aab507cc26f5aea10af8afa9851b-v300g00","3/1/2015","10:16:38","Anonymous@Anonymous.invalid","0873529097","087","76","0.266","0.21"
"SDg7je101-b2282f1118686ee39719d0a53c00c299-v300g00","3/1/2015","13:39:08","0114358297","0873546275","087","14","0.049","0.21"
"SDn7kq101-b04a267da7bb1b8f29f327fe1423538f-v300g00","3/1/2015","16:52:14","0114358297","0873546275","087","6","0.021","0.21"
"SD0gb4301-695accdb4ff6a9e26be6fc520fa893c3-v300g00","3/1/2015","15:56:57","0114358297","0873546275","087","6","0.021","0.21"
"SDv6n8201-2634518bc8d5df434f4582835a81e27f-v300g00","3/1/2015","09:30:52","Unavailable","0873534205","087","3","0.0105","0.21"
"SDivio201-647a46bc70eda17d0726a8b008eb1156-v300g00","3/1/2015","15:14:03","0114358311","0873546275","087","13","0.0455","0.21"
"SDb7cm101-33fcd3ab871cca2b9f421767af460877-v300g00","3/1/2015","14:42:47","0114358297","0873546275","087","8","0.028","0.21"
"SDkvqc101-3ad4713540402023523dd81da8503eac-v300g00","3/1/2015","10:51:56","Unavailable","0873546275","087","14","0.049","0.21"
"SD20ie201-4d596b87b1684d10d0d15673395c6f6f-v300g00","3/1/2015","16:00:10","0114358311","0873546275","087","11","0.0385","0.21"
"SD20ps001-3a730e8cf102a792219141497c85b797-v300g00","3/1/2015","12:30:07","0114358297","0873546275","087","5","0.0175","0.21"
"SDl7fk101-5d5374409dce12e79d6dee0f40ff329d-v300g00","3/1/2015","17:17:50","0114358311","0873546275","087","14","0.049","0.21"
"SD57hg201-98196ea08a3fac94aa7029767b85155a-v300g00","3/1/2015","13:03:17","0114358311","0873546275","087","14","0.049","0.21"
"SDenea301-8242e990ea455b06fe8779d581a0f0fe-v300g00","3/1/2015","14:06:55","0114358311","0873546275","087","10","0.035","0.21"
"SDumga101-2d768591eb96b411949838bfeca18eea-v300g00","3/1/2015","15:17:46","0114358297","0873546275","087","14","0.049","0.21"
"SDmfhc201-84d4a6e39b5410bc64d556419fdc7bfa-v300g00","3/1/2015","13:51:18","0114358311","0873546275","087","14","0.049","0.21"
"SD20bk101-3a3884a936427fb9f039e803d976fdc5-v300g00","3/1/2015","15:59:20","0114358297","0873546275","087","9","0.0315","0.21"
"SD7serd01-6afc6a5202347b24cdfad11c89507cd3-v300g00","3/1/2015","07:38:15","0219188500","0214881822","D850","194","0.679","0.21"
"SD7k95d01-532a804796ec110cc14adb01aa93dba3-v300g00","3/1/2015","08:34:13","0219188500","0214881822","D850","187","0.6545","0.21"
"SDhpd5401-9ece59dc2c72e47bb50ab8067d21bac2-v300g00","3/1/2015","08:48:06","0874901777","0112157000","D850","114","0.399","0.21"

This is the logic that i want my output to follow.

Everything that is in File A and matches FileB should be placed in file C. File C should have all the columns that are in File B.

I hope this makes sense.

Regards,

Paxley

2

Frequently asked question, frequently given answer:

fgrep -f FILEA FILEB > FILEC

Faster and more precise: search only in the first column:

awk -F, 'NR==FNR {A[$0]; next} {for (a in A) if (index($1,a)) {print; next}}' FILEA FILEB > FILEC
3

Thank you MadeInGermany,

fgrep -f FILEA FILEB > FILEC

-- somehow this does not work

awk -F, 'NR==FNR {A[$0]; next} {for (a in A) if (index($1,a)) {print; next}}' FILEA FILEB > FILEC

-- this is not that fast i have been waiting for the command to complete for the last 5 minutes and the command is loves memory between 80-90% CPU usage.

awk -F, 'NR==FNR {A[$0]; next} {for (a in A) if (index($1,a)) {print; next}}' mweb_mediation.csv mweb_rev1.csv > mweb_inbound_march2015.csv

-- that is how I ran my command

4

awk 'FNR==NR {fa[$1];next} $2 in fa' FILEA FS='"' FILEB > FILEC