List of common identifiers

Shell Programming and Scripting
1

Hi all,

I have 4 file and I want to find the common identifier in each file. For example:

FILE1
goat
door
bear
cat

FILE2
goat
moose
dog
cat

FILE3
goat
yak
tiger
cat

FILE4
goat
yak
lion
cat

So its obvious that goal and cat are common for all 4 files so that would get printed out into an output file.

Is there a quick UNIX command for that?

thanks

2 
$ 
$ cat file1
goat
door
bear
cat
$ cat file2
goat
moose
dog
cat
$ cat file3
goat
yak
tiger
cat
$ cat file4
goat
yak
lion
cat
$ awk '{x[$0]=x[$0]FILENAME} END{for (i in x){if (x=="file1file2file3file4"){print i}}}' file1 file2 file3 file4
cat
goat
$

tyler_durden

---------- Post updated at 10:13 PM ---------- Previous update was at 10:00 PM ----------

$ 
$ sort file1 >sorted_file1; sort file2 >sorted_file2
$ sort file3 >sorted_file3; sort file4 >sorted_file4
$ join sorted_file1 sorted_file2 >common_12
$ join sorted_file3 sorted_file4 >common_34
$ join common_12 common_34
cat
goat
$

tyler_durden

---------- Post updated at 10:17 PM ---------- Previous update was at 10:13 PM ----------

$ 
$ sort file1 file2 file3 file4 | uniq -c | awk '$1==4 {print $2}'
cat
goat
$ 
$

tyler_durden

3 
cat file? | awk ' { word[$1]++ }
END {
       for (i in word) {
           print i,word
           }
      }
' | sort -k 2,2nr

If you like to get top two, then add after sort

|  head -2

And if you like to save result to file, add to the end of commandline

 > result.txt
4

This should work with variable number of input files:

(use gawk, nawk or /usr/xpg4/bin/awk on Solaris)

awk > outfile 'END { 
  for (W in w) if (w[W] == ARGC - 1) print W 
	}
{  f[$0,FILENAME]++ || w[$0]++ }
' infile1 [infile2 ... infilen]
5

A vintage unix approach The first "cat" command is not needed but it leaves the script looking neat!

cat file1 | \
        cat - file2 | sort | uniq -d | \
        cat - file3 | sort | uniq -d | \
        cat - file4 | sort | uniq -d

cat
goat
6

Hm,
try your script with a pattern that appears more than once in the same input file, but that is not present in all the input files.

7

Thanks, that's a good point, radoulov.

This script:

sort file1 file2 file3 file4 | uniq -c | awk '$1==4 {print $2}'

would not solve the OP's problem in that case.

tyler_durden