Hi.
In thread Grouping and counting rovf and I had a mini-discussion on grep
and awk
.
Here is a demo script that compares the awk
and grep
approaches for this single problem:
#!/usr/bin/env bash
# @(#) s2 Demonstrate group and count with standard commands.
# Utility functions: print-as-echo, print-line-with-visual-space, debug.
# export PATH="/usr/local/bin:/usr/bin:/bin"
LC_ALL=C ; LANG=C ; export LC_ALL LANG
pe() { for _i;do printf "%s" "$_i";done; printf "\n"; }
pl() { pe;pe "-----" ;pe "$*"; }
db() { ( printf " db, ";for _i;do printf "%s" "$_i";done;printf "\n" ) >&2 ; }
db() { : ; }
C=$HOME/bin/context && [ -f $C ] && $C awk grep cut sort uniq
FILE=${1-data1}
pl " Input data file $FILE:"
cat $FILE
pl " Results, awk filter:"
awk -F'|' '$4 == "\""1"\"" && $5=="\"""Y""\"" ' $FILE |
cut -f8 -d'|' | sort | uniq -c
pl " Results, grep filter:"
grep -E '^([^|]+[|]){3}.1...Y' $FILE |
cut -f8 -d'|' | sort | uniq -c
exit 0
producing:
$ ./s2
Environment: LC_ALL = C, LANG = C
(Versions displayed with local utility "version")
OS, ker|rel, machine: Linux, 3.16.0-4-amd64, x86_64
Distribution : Debian 8.4 (jessie)
bash GNU bash 4.3.30
awk GNU Awk 4.1.1, API: 1.1 (GNU MPFR 3.1.2-p3, GNU MP 6.0.0)
grep (GNU grep) 2.20
cut (GNU coreutils) 8.23
sort (GNU coreutils) 8.23
uniq (GNU coreutils) 8.23
-----
Input data file data1:
1111|"ZZZ"|"Date"|"1"|"Y"|"ABC"|""|CC
1234|"ZZZ"|"Date"|"1"|"Y"|"ABC"|""|AA
ABCD|"ZZZ"|"Date"|"1"|"Y"|"ABC"|""|AA
EFGH|"ZZZ"|"Date"|"1"|"Y"|"ABC"|""|BB
IJKLM|"ZZZ"|"Date"|"1"|"Y"|"ABC"|""|BB
NOPQ|"ZZZ"|"Date"|"1"|"Y"|"ABC"|""|BB
foox|"YYY"|"Date"|"2"|"Y"|"ABC"|""|BB
barx|"YYY"|"Date"|"1"|"N"|"ABC"|""|BB
bazx|"YYY"|"Date"|"3"|"N"|"ABC"|""|CC
-----
Results, awk filter:
2 AA
3 BB
1 CC
-----
Results, grep filter:
2 AA
3 BB
1 CC
Both outputs are the same. The suggestion of rovf is quite correct, as illustrated, you can use grep
to match field content. However, it also illustrates why I do not usually do that. Both commands are more than really basic, perhaps intermediate-complex. The grep
makes use of a regular expression that is beyond many people's ability to understand it, much less create it. The awk
is also not basic, but the form suggests what we are looking for, namely the content of fields 4 and 5. The extra "baggage" is to take care of the quotes. There are probably other ways of doing that, but this one seems straight-forward when one realizes how to quote an escaped quote.
So from a results-oriented POV, the solutions are the same.
However, an additional point is that, if one uses awk
as the first step, one might also make the entire solution awk-based. I used standard commands because the problem decomposed easily into steps, until the extra requirements of of field contents arose.
I don't think the issue of speed would come up, but generally grep
is somewhat fast than awk
in similar situations. I have posted elsewhere some results comparing utilities and languages for matching.
So, rovf is correct, but the issue of transparency is more important, in my opinion.
Thanks to rovf for pointing our that grep
and regular expressions are very powerful ... cheers, drl