You are right. How big were your files? Only OS and system load would make difference. I want to try it on Solaris 10. In the case of large files, there would be a threshold at which awk could gain advantage. It all depends on how these utilities are written. The multiple commands have smaller memory footprint but use multiple redirections (waiting for I/O).
$ wc -l file1 file2
13872 file1
171942 file2
185814 total
As for the threshold, I really have no idea but as the Unix saying goes: "do one thing and do it well". And, again, awk is the right tool for the job. It is unbeatable when it get to processing data files. Don't be afraid by its syntax and paradigm and have a look here:
awk.info
Your script gives syntax error on Solaris 10.
I don't have time to look into it right now.
Use nawk or /usr/xpg4/bin/awk on Solaris.
Thanks. nawk does it.
I looked at timex on nawk and command line. It is not really conclusive. Time varies wildly:
nawk
first time
real 0.58
user 0.19
sys 0.06
second time
real 0.35
user 0.18
sys 0.05
modified script (second version, for loop and grep, writing to 2 files)
for i in `cat file11 file22 | cut -f1 -d \| | sort | uniq -u`
do
echo $i
grep -h $i file1 >> fil1
grep -h $i file2 >> fil2
done
first time
real 0.63
user 0.01
sys 0.00
second time
real 0.57
user 0.42
sys 0.09
Two files were as follows:
# wc -c file11 file22
1574508 file1
1148400 file2
2722908 total
My conclusion:
These tests are not really rigorous and depend on other activities on the system.
The gain in performance is really negligible. Both approaches are fine from performance point of view. Using unix separate commands is much easier to learn and scripts are much easier to maintain. If not fluent in awk and needing more powerful language learning perl is probably a better idea. Awk syntax is not very friendly and I think it is a legacy program.
Thanks Franklin52 for taking your time to explain this newbie. I'm still learning the code you have shared. Can I have some more questions on this please.
I'm able to understand that only while reading the first file NR and FNR will be equal. Is this some kind of "if condition". I could not see any "if" there.
Kindly bear my ignorance.
About the solution by gcp, it is self explanatory and I have no questions on it.
It just defines an array variable and the command:
$1 in a
checks if the variable has been defined.
Regards