Argument list too long problem

Shell Programming and Scripting
1

I have a huge set of files (with extension .common) in my directory around 2 million. When I run this script on my Linux with BASH, I get /bin/awk: Argument list too long

awk -F'\t' '
  NR == FNR { a[$1]=NR }
  NR != FNR {
    sub(".common", "", FILENAME)
    print a[$2], FILENAME,  $1
  }
' dictionary.dit *.common

Any idea how can I fix this? One way is to put the non-overlapping files in two or three directories and then run the above script but I am looking for a better solution.

2

Try:

find DIR -name '*.common' | xargs awk ... dictionary.dit