Explain the AWK syntax in detail(NR,FNR,NF)

Shell Programming and Scripting
1

I am learning file comparison using awk.

I found syntax like below,

awk '(NR == FNR) { s[$0]; next } { for (i=1; i<=NF; i++) if ($i in s) delete s[$i] } END { for (i in s) print i }' tests tests2

I couldn't understand what is the Syntax ...Can you please explain in detail?

What exactly does it do?

2

If you google "GNU Awk User Guide", you will find an excellent tutorial and reference guide, with many examples, a gentle introduction, and Contents and Index cross-references. Highly commended, but I can't read it for you -- you need to absorb what you need for yourself.

Edit: My favourite quote from that Guide is:

I can explain it for you, but I can’t understand it for you.

But my explanations would not be as good as the Guide, which has almost 20 years of work behind it.

I can point you to the parts of the Guide most relevant to this code fragment. There is no purpose in me pasting sections of it into answers. Would you want me to play your adventure game for you? My adventure game is technology, and I have been playing it for 50 years. Hope you have the same joy I have had.

Some awk variables are automatically set as records are read. The Summary of variables FNR, NR and NF is in section 4.14.

There is a well-known trick here which is not explained in the Guide. As FNR counts lines in each file and NR counts total lines, so (NR == FNR) is a test that you are currently reading the first file named in the arguments.

Knowing what NF is, you should be able to see what the for construct does.

The overall design is that we save all the words from file 1 (in an array s[]), discard the entries which we see in any following files, and list what is left. You need to check the purpose of the END pattern too.

3

Here is same with example data. Maybe easier to "read".
I'll say that it's possible easier / faster to understand if someone try to explain. Totaltime is maybe less and you can use the saved time to ... ex. to great something new using this example ...

Make some.sh and execute it. Paul already has written the most important: read guide and meaning of FNR and NR.

cat <<EOF > tests
1234
2345
67890
3456
78901
66775
4567
5678
EOF
cat <<EOF > tests2
1234 56789
2345 67890
88888 3456
89012 4567
5678 99999
EOF

awk '
NR == FNR     {  # read 1st file and save lines to the array s - unique "some values"
                            # little "guru" method to test line from 1st file or not
                        s[$0]
                        next # read next line
                }
                {   # remove those values from array which are in 2nd file in some field
                    # done for every line in 2nd file, because previous rule process 1st file
                        for (i=1; i<=NF; i++)
                                if ($i in s) delete s[$i]
                }
END {
        for (i in s) # values which was only in 1st file
                print i
        }
' tests tests2