Using Bash scripting to compare arrays looking for matches

Shell Programming and Scripting
1

I have two arrays I need to compare against a third, looking for matches, not differences. I think I'm going to have to convert the arrays to files and grep them, but I'm not too sure if there's a tool to enable me to matches specifically, instead of differences.

Thanks in advance!

Karl

2

Why cannot you simply iterate through arrays and 'compare'?

3

I was not aware that compare allowed only output of matches... Also, I'm afraid I'm not sure how I would iterate through an array.

My idea so far was to take the third array (shortlist) and compare it to the others one at a time by using the list as an input to grep. Conceptually:

Generate two long lists
Input into arrays
Generate short list
Input into array
Feed short list into grep and compare against the first long list
-No matches? Go on.

  • Matches? Print and go on.
    Compare against the second list
    -Matches? Print and go on.
    -No matches? Go on.

I have a script so far that generates the arrays from a few different files.

I guess I just need to figure out how to input a file, line by line, into grep so that grep would use the first line, and look for it in the array (or a file? can grep check arrays?) then take the second line, etc.

4

you might not need arrays at all. Why don't you take a 'first long' list and a 'short list' and find the 'matches' - do the same with the 'second long list'.
If you post samples (short list and a one long list) and describe the 'matching' algorithm, we might help to get started.
Please use code tags when posting data samples. Also provide the desired result based on your sample input.

5

So, let's see if I can construct an example.

Long list 1 contains a list of files, so does long list 2. These filenames are being compared to shortlist, which is a list generated from a find command used on a directory. The long lists contain files that ARE NOT allowed in the directory.

I had no idea what algorithm to use to look for matches; that was a large part of my question.

Long_List1=( $( cat /lists/badfiles ) )
Long_List2=( $( cat /lists/disallowedfiles.txt ) )
find /files -name *.stuff > /tmp/stuff.txt
short_list=( $( cat /tmp/stuff.txt ) )

I was thinking of either skipping reading the files into arrays and using short_list (stuff.txt) as input into grep, or using the arrays in the fashion specified above.

6

depending on the actual content of files, here's one way:

grep -f /tmp/stuff.txt /lists/badfiles
grep -f /tmp/stuff.txt /lists/disallowedfiles.txt
7

Thanks! Contents of the file is just a list of files, one per line:

badfiles.txt:

file.stuff
file2.stuff
file3.stuff

One of the long lists also has semicolons at the end of the line, and it has more information. I'll be needing to strip out the semicolons and things afterwards, but I need to compare first.

EDIT: Forgot to mention, I'm on a strange Solaris platform, the -f switch doesn't work for grep.

Partly the issue of why I'm asking for help. I'm used to a BSD/Mac os x platform, and Solaris is ridiculously wonky. Well, okay, maybe it's not so bad, I'm just not used to it... :rolleyes:

8

Please provide a sample of a 'long list file' within code tags.
Does the provided 'grep' suggestion work?

9

The sample provided is the long list file, simply cat'ed from a .db to a .txt.

Unfortunately the grep suggestion failed... perhaps I can cat the filelist from the directory into xargs and from there feed it into grep?

10

an awk alternative:

nawk '
  FNR==NR {f1[".*" $0 ".*"];next }
  { for( i in f1 ) if ($0 ~ i) { print; break} }' /tmp/stuff.txt /lists/badfiles