One instance of comparing grep and awk

Hi.

In thread Grouping and counting rovf and I had a mini-discussion on grep and awk .

Here is a demo script that compares the awk and grep approaches for this single problem:

#!/usr/bin/env bash

# @(#) s2       Demonstrate group and count with standard commands.

# Utility functions: print-as-echo, print-line-with-visual-space, debug.
# export PATH="/usr/local/bin:/usr/bin:/bin"
LC_ALL=C ; LANG=C ; export LC_ALL LANG
pe() { for _i;do printf "%s" "$_i";done; printf "\n"; }
pl() { pe;pe "-----" ;pe "$*"; }
db() { ( printf " db, ";for _i;do printf "%s" "$_i";done;printf "\n" ) >&2 ; }
db() { : ; }
C=$HOME/bin/context && [ -f $C ] && $C awk grep cut sort uniq

FILE=${1-data1}

pl " Input data file $FILE:"
cat $FILE

pl " Results, awk filter:"
awk -F'|' '$4 == "\""1"\"" && $5=="\"""Y""\"" ' $FILE |
cut -f8 -d'|' | sort | uniq -c

pl " Results, grep filter:"
grep -E '^([^|]+[|]){3}.1...Y' $FILE |
cut -f8 -d'|' | sort | uniq -c

exit 0

producing:

$ ./s2

Environment: LC_ALL = C, LANG = C
(Versions displayed with local utility "version")
OS, ker|rel, machine: Linux, 3.16.0-4-amd64, x86_64
Distribution        : Debian 8.4 (jessie) 
bash GNU bash 4.3.30
awk GNU Awk 4.1.1, API: 1.1 (GNU MPFR 3.1.2-p3, GNU MP 6.0.0)
grep (GNU grep) 2.20
cut (GNU coreutils) 8.23
sort (GNU coreutils) 8.23
uniq (GNU coreutils) 8.23

-----
 Input data file data1:
1111|"ZZZ"|"Date"|"1"|"Y"|"ABC"|""|CC
1234|"ZZZ"|"Date"|"1"|"Y"|"ABC"|""|AA
ABCD|"ZZZ"|"Date"|"1"|"Y"|"ABC"|""|AA
EFGH|"ZZZ"|"Date"|"1"|"Y"|"ABC"|""|BB
IJKLM|"ZZZ"|"Date"|"1"|"Y"|"ABC"|""|BB
NOPQ|"ZZZ"|"Date"|"1"|"Y"|"ABC"|""|BB
foox|"YYY"|"Date"|"2"|"Y"|"ABC"|""|BB
barx|"YYY"|"Date"|"1"|"N"|"ABC"|""|BB
bazx|"YYY"|"Date"|"3"|"N"|"ABC"|""|CC

-----
 Results, awk filter:
      2 AA
      3 BB
      1 CC

-----
 Results, grep filter:
      2 AA
      3 BB
      1 CC

Both outputs are the same. The suggestion of rovf is quite correct, as illustrated, you can use grep to match field content. However, it also illustrates why I do not usually do that. Both commands are more than really basic, perhaps intermediate-complex. The grep makes use of a regular expression that is beyond many people's ability to understand it, much less create it. The awk is also not basic, but the form suggests what we are looking for, namely the content of fields 4 and 5. The extra "baggage" is to take care of the quotes. There are probably other ways of doing that, but this one seems straight-forward when one realizes how to quote an escaped quote.

So from a results-oriented POV, the solutions are the same.

However, an additional point is that, if one uses awk as the first step, one might also make the entire solution awk-based. I used standard commands because the problem decomposed easily into steps, until the extra requirements of of field contents arose.

I don't think the issue of speed would come up, but generally grep is somewhat fast than awk in similar situations. I have posted elsewhere some results comparing utilities and languages for matching.

So, rovf is correct, but the issue of transparency is more important, in my opinion.

Thanks to rovf for pointing our that grep and regular expressions are very powerful ... cheers, drl

4 Likes

I prefer awk, because I know it well, so there is little effort to make it precise and efficient.
For extreme robustness (e.g. long input lines) I go with perl. But usually takes longer till it does what I want.
For a "work on fields" task grep is often unprecise, or the readability and extend-ability are worse.