Continued trouble matching fields in different files and selective field printing ([g]awk)

I apologize in advance, but I continue to have trouble searching for matches between two files and then printing portions of each to output in [g]awk and would very much appreciate some help.

I have data as follows:

File1

PS012,002 PRQ                 0   1  1 17  1  0 -1    3  2  1  2    -1   1   1  -1      -1      -1      -1    0  501     0
PS012,002 MRJ>                0   3 -1 -1 -1  1 -1   -1 -1  0  0     2   3   3   2      -1      -1      -1    0  562     0
PS012,002 MVL                 0   5 -1 -1 -1 -1 -1   -1 -1 -1 -1    -1   5   0  -1      -1      -1      -1   -1   -1    -1
PS012,002 D                  -1   5 -1 -1 -1 -1 -1   -1 -1 -1 -1    -1   6   6  -1      -1      -1      -1    0  509     0
PS012,002 GMR                 0   1  0 17  1  0 -1    2  3  1  2    -1   1   1  -1      -1      -1      -1    0  501     0
PS012,002 VB                  0  13 -1 -1 -1  1 -1   -1 -1  0  0     3   2   2   2      -1      -1      -1    0  502     0
PS012,002 W                   0   6 -1 -1 -1 -1 -1   -1 -1 -1 -1    -1   6   6  -1      -1      -1      -1    0  509     0
PS012,002 BVL                 0   1  0 17 12  0 -1    2  3  1  1    -1   1   1  -1      -1      -1      -1    0  501     0
PS012,002 HJMNW               0   2 -1 -1 -1 11 -1   -1 -1  1  1     3   2   2   2      -1      -1      -1    0  502     0
PS012,002 MN                  0   5 -1 -1 -1 -1 -1   -1 -1 -1 -1    -1   5   0  -1      -1      -1      -1   -1   -1    -1
PS012,002 >R<                 0   2 -1 -1 -1  1 -1   -1 -1  0  1     3   2   5   2      -1      -1      -1    0  505     0

File2

008  PS012,001 CMJNJ      008 ?         
010  PS012,001 MZMWR      010 ?         
011  PS012,001 L          011 ?         
012  PS012,001 DWD        012 ?         
014  PS012,002 JC<        014 PRQ       
016  PS012,002 JHWH       016 MRJ>      
018  PS012,002 ?          018 MVL       
019  PS012,002 KJ         019 D         
020  PS012,002 GMR        020 GMR       
021  PS012,002 XSJD       021 VB        
023  PS012,002 KJ         023 W         
024  PS012,002 PSS        024 BVL       
025  PS012,002 >MWN       025 HJMNW     
026  PS012,002 MN         026 MN        
027  PS012,002 BN         027 >R<       
028  PS012,002 >DM        028 ?         

If $2 and $5 of File2 match $1 and $2 of File1, I would like to print $4 of File2 followed by $0 of File1.

Intended Output:

014 PS012,002 PRQ                 0   1  1 17  1  0 -1    3  2  1  2    -1   1   1  -1      -1      -1      -1    0  501     0
016 PS012,002 MRJ>                0   3 -1 -1 -1  1 -1   -1 -1  0  0     2   3   3   2      -1      -1      -1    0  562     0
018 PS012,002 MVL                 0   5 -1 -1 -1 -1 -1   -1 -1 -1 -1    -1   5   0  -1      -1      -1      -1   -1   -1    -1
019 PS012,002 D                  -1   5 -1 -1 -1 -1 -1   -1 -1 -1 -1    -1   6   6  -1      -1      -1      -1    0  509     0
020 PS012,002 GMR                 0   1  0 17  1  0 -1    2  3  1  2    -1   1   1  -1      -1      -1      -1    0  501     0
021 PS012,002 VB                  0  13 -1 -1 -1  1 -1   -1 -1  0  0     3   2   2   2      -1      -1      -1    0  502     0
023 PS012,002 W                   0   6 -1 -1 -1 -1 -1   -1 -1 -1 -1    -1   6   6  -1      -1      -1      -1    0  509     0
024 PS012,002 BVL                 0   1  0 17 12  0 -1    2  3  1  1    -1   1   1  -1      -1      -1      -1    0  501     0
025 PS012,002 HJMNW               0   2 -1 -1 -1 11 -1   -1 -1  1  1     3   2   2   2      -1      -1      -1    0  502     0
026 PS012,002 MN                  0   5 -1 -1 -1 -1 -1   -1 -1 -1 -1    -1   5   0  -1      -1      -1      -1   -1   -1    -1
027 PS012,002 >R<                 0   2 -1 -1 -1  1 -1   -1 -1  0  1     3   2   5   2      -1      -1      -1    0  505     0

I have been attempting the following code to no avail.

awk 'NR==FNR {q=$1 SUBSEP $2; A[q]=$0; next} ($2,$5) in A{print $0,A[$1,$2]}' file1 file2

Close. Note that in the command:

awk '
NR==FNR {q=$1 SUBSEP $2; A[q]=$0; next}
($2,$5) in A{print $0,A[$1,$2]}' file1 file2

the NR==FNR condition section is gathering data from the first file where you are looking at fields $1 and $2 . So that line looks OK, but since you aren't modifying either of those fields, you don't need the q variable to keep those values (like you did in your last thread) so that line can be written more simply as:

NR==FNR {A[$1,$2]=$0; next}

In the other line of code, you're looking at fields $2 and $5 from the 2nd input file. You got that right in one case, but you didn't in the other case. If you change:

($2,$5) in A{print $0,A[$1,$2]}

to:

($2,$5) in A{print $0,A[$2,$5]}

you should get what you want.

Thank you so much for this Don and also for your (and others') patience as I posted two very similar threads that kept tripping me up (I continue to have trouble with this sort of text manipulation).

I may be wrong, but in the code you suggest,

{print $0,A[$2,$5]}

it looks like that will print all of File2 (i.e., the entire line) prior to the matched output.

Perhaps

{print $1,A[$2,$5]}

does the trick.

Sorry about that. I didn't look closely enough at your output spec after noticing the wrong fields being used as indexes into your A[] array. If we go back to your 1st post in this thread, you said:

so instead of:

{print $1,A[$2,$5]}

it would seem that you might need:

{print $4,A[$2,$5]}

And, of course, you still need the condition on that action:

($2,$5) in A{print $4,A[$2,$5]}

Please don't be sorry. I appreciate your help more than you know.

---------- Post updated at 08:38 PM ---------- Previous update was at 07:00 PM ----------

Don, if I may, your code help works terrific except I'm having issues when the records of $1 and $2 are repeated somewhere else in the field.

For example, there is a section in my data that looks like this portion of File1:

File1

PS012,007 DKJ                 0   1  0 17  1  1 -1   62 -1  0  0     3  13  13   2      -1      -1      -1    0  521     0
PS012,007 S>M                 0   2 -1 -1 -1  1 -1   -1 -1  0  2     3   2   0  -1      -1      -1      -1   -1   -1    -1
PS012,007 GBJ                 0   1  0 17  1  1 -1   62 -1  0  0     3  13   2  -1      -1      -1      -1    0  502     0
PS012,007 D                  -1   5 -1 -1 -1 -1 -1   -1 -1 -1 -1    -1   6   6  -1      -1      -1      -1    0  519     0
PS012,007 BXR                 0   1  0 17  1  1 -1   62 -1  0  0     2   1   1  -1      -1      -1      -1    0  521     0
PS012,007 B                   0   5 -1 -1 -1 -1 -1   -1 -1 -1 -1    -1   5   0  -1      -1      -1      -1   -1   -1    -1
PS012,007 >R<                 0   2 -1 -1 -1  1 -1   -1 -1  0  1     3   2   5   2      -1      -1      -1    0  505     0
PS012,007 W                   0   6 -1 -1 -1 -1 -1   -1 -1 -1 -1    -1   6   6  -1      -1      -1      -1    0  509     0
PS012,007 DKJ                 0   1  6 18  1  1 -1   62 -1  0  0     2   1   1  -1      -1      -1      -1    0  521     0

Here, there are two instances where $1=="PS12,007" and $2=="DKJ" (the first and last lines in the example, but these are not first and last in the actual file). However there is different data in fields $3-$23.

When I run the code you helped me with:

awk 'NR==FNR {A[$1,$2]=$0; next} ($2,$5) in A{print $1,A[$2,$5]}' file1 file2

with the following portion of File2:

120  PS012,007 ?          120 DKJ       
122  PS012,007 KSP        122 S>M       
123  PS012,007 ?          123 GBJ       
125  PS012,007 ?          125 D         
126  PS012,007 YRP        126 BXR       
127  PS012,007 B          127 B         
128  PS012,007 <LJL       128 >R<       
129  PS012,007 L          129 ?         
130  PS012,007 H          130 ?         
131  PS012,007 >RY        131 ?         
133  PS012,007 ?          133 W         
134  PS012,007 ZQQ        134 DKJ       

I get this output:

120  PS012,007 DKJ                 0   1  6 18  1  1 -1   62 -1  0  0     2   1   1  -1      -1      -1      -1    0  521     0
122  PS012,007 S>M                 0   2 -1 -1 -1  1 -1   -1 -1  0  2     3   2   0  -1      -1      -1      -1   -1   -1    -1
123  PS012,007 GBJ                 0   1  0 17  1  1 -1   62 -1  0  0     3  13   2  -1      -1      -1      -1    0  502     0
125  PS012,007 D                  -1   5 -1 -1 -1 -1 -1   -1 -1 -1 -1    -1   6   6  -1      -1      -1      -1    0  519     0
126  PS012,007 BXR                 0   1  0 17  1  1 -1   62 -1  0  0     2   1   1  -1      -1      -1      -1    0  521     0
127  PS012,007 B                   0   5 -1 -1 -1 -1 -1   -1 -1 -1 -1    -1   5   0  -1      -1      -1      -1   -1   -1    -1
128  PS012,007 >R<                 0   2 -1 -1 -1  1 -1   -1 -1  0  1     3   2   5   2      -1      -1      -1    0  505     0
133  PS012,007 W                   0   6 -1 -1 -1 -1 -1   -1 -1 -1 -1    -1   6   6  -1      -1      -1      -1    0  509     0
134  PS012,007 DKJ                 0   1  6 18  1  1 -1   62 -1  0  0     2   1   1  -1      -1      -1      -1    0  521     0

Here, fields $4-$24 are identical when fields $1 and $2 of File1 have a repeated string. However, fields $4-$24 should remain as they appear in File1 (i.e., fields $3-$23 in File1). The end result should look like this:

120  PS012,007 DKJ                 0   1  0 17  1  1 -1   62 -1  0  0     3  13  13   2      -1      -1      -1    0  521     0
122  PS012,007 S>M                 0   2 -1 -1 -1  1 -1   -1 -1  0  2     3   2   0  -1      -1      -1      -1   -1   -1    -1
123  PS012,007 GBJ                 0   1  0 17  1  1 -1   62 -1  0  0     3  13   2  -1      -1      -1      -1    0  502     0
125  PS012,007 D                  -1   5 -1 -1 -1 -1 -1   -1 -1 -1 -1    -1   6   6  -1      -1      -1      -1    0  519     0
126  PS012,007 BXR                 0   1  0 17  1  1 -1   62 -1  0  0     2   1   1  -1      -1      -1      -1    0  521     0
127  PS012,007 B                   0   5 -1 -1 -1 -1 -1   -1 -1 -1 -1    -1   5   0  -1      -1      -1      -1   -1   -1    -1
128  PS012,007 >R<                 0   2 -1 -1 -1  1 -1   -1 -1  0  1     3   2   5   2      -1      -1      -1    0  505     0
133  PS012,007 W                   0   6 -1 -1 -1 -1 -1   -1 -1 -1 -1    -1   6   6  -1      -1      -1      -1    0  509     0
134  PS012,007 DKJ                 0   1  6 18  1  1 -1   62 -1  0  0     2   1   1  -1      -1      -1      -1    0  521     0

I've attempted various code variations that take what you had helped me with and integrates it with count and/or loop functions in awk. None of these have been successful.

If you are using only $1 & $2 as a key into file1 to match against $2 & $5 as a key into file2 and there those keys do not uniquely identify which record is to be processed when the data associated with those keys are not unique, there isn't anything I can do to help you. When there are multiple records with the same keys, you need to clearly describe how your code is supposed to determine which of those records are supposed to be paired.

The way the code is written now, the last record with a given key in file1 will be matched with all records with that key in file2 . We now know that that isn't what you want. How do you uniquely describe a key that can be used to get what you do want? Or, in other words, what do you want to add to your current keys so your keys will uniquely identify the records you want to match in both files?

The fact that combinations of $2 and $5 can occur multiple times is a new condition that was not apparent in the first post. So what is the criterion for what goes with what? Is it the order in which they appear in both files or does $3 in file2 play a role somehow?

Assuming it is the former of these two possibillties, try this modification:

awk 'NR==FNR {A[$1,$2,C1[$1,$2]++]=$0; next} ($2,$5,0) in A{print $1,A[$2,$5,C2[$2,$5]++]}' file1 file2

which (with the new sample in post #5) produces:

120 PS012,007 DKJ                 0   1  0 17  1  1 -1   62 -1  0  0     3  13  13   2      -1      -1      -1    0  521     0
122 PS012,007 S>M                 0   2 -1 -1 -1  1 -1   -1 -1  0  2     3   2   0  -1      -1      -1      -1   -1   -1    -1
123 PS012,007 GBJ                 0   1  0 17  1  1 -1   62 -1  0  0     3  13   2  -1      -1      -1      -1    0  502     0
125 PS012,007 D                  -1   5 -1 -1 -1 -1 -1   -1 -1 -1 -1    -1   6   6  -1      -1      -1      -1    0  519     0
126 PS012,007 BXR                 0   1  0 17  1  1 -1   62 -1  0  0     2   1   1  -1      -1      -1      -1    0  521     0
127 PS012,007 B                   0   5 -1 -1 -1 -1 -1   -1 -1 -1 -1    -1   5   0  -1      -1      -1      -1   -1   -1    -1
128 PS012,007 >R<                 0   2 -1 -1 -1  1 -1   -1 -1  0  1     3   2   5   2      -1      -1      -1    0  505     0
133 PS012,007 W                   0   6 -1 -1 -1 -1 -1   -1 -1 -1 -1    -1   6   6  -1      -1      -1      -1    0  509     0
134 PS012,007 DKJ                 0   1  6 18  1  1 -1   62 -1  0  0     2   1   1  -1      -1      -1      -1    0  521     0

==edit:==
The above code can create problems if it is possible combinations only appear once in file1 and twice in file2, which would lead to half lines being printed.
In that case it could be modified to this:

awk 'NR==FNR {A[$1,$2,++C1[$1,$2]]=$0; next} ($2,$5,++C2[$2,$5]) in A{print $1,A[$2,$5,C2[$2,$5]]}' file1 file2

So that would mean that a second occurrence in file2 would not get printed if it only appears once in file 1

Sorry Don, as I am so new to this I did not realize that a repeat key item would cause this behavior until after I had tested the code you helped me modify. I thought (erroneously) that the order in which they appear (as Scrutinizer alludes to below) would be enough to disambiguate the two items in the array. As someone not from a math/science background (I'm in the Humanities) I do not fully understand the behavior of associative arrays yet (I understand the concept, but how to bring that to bear in coding has been a challenge for me). Nevertheless, I really appreciate your help.

---------- Post updated at 09:05 AM ---------- Previous update was at 09:04 AM ----------

Thank you Scrutinizer so much for this. I will have to test it further in a bit, but an initial run looks very promising. Ironically, I began this thread attempting to introduce a unique field in File1 as an additional, disambiguating search criteria for a part of another text processing need I had that was running aground due to repeat items in a field. In that other process I attempted multiple incrementing and count functions to no avail. I believe that your help in the proper use of the increment op within this code will help me solve that other issue as well. I appreciate your help very much.