Gawk and regexp

Shell Programming and Scripting
1

Hello,

This is a problem I've worked on a while and can't figure out.

There is a file.txt

..some stuff..
[[Category:98 births]]
[[Category:2nd C. deaths]]
..some stuff..

The Awk program is trying to extract the year portion of the birth and death ("98: and "2nd C.") using the below technique

#!/bin/awk

@include filefuncs

BEGIN{

  fp = readfile("file.txt")

  birth = years(fp, "births")
  death = years(fp, "deaths")

  print "Birth = "birth
  print "Death = "death

}
function years(fp, type     ,a,b)
{
  if(type == "births") {
    if(match(fp,/\yCategory:[[:punct:]A-z0-9]* births\y/, a)) 
      split(a[0],b,":|births")
      return b[2]
 }
 if(type == "deaths") {
   if(match(fp,/\yCategory:[[:punct:]A-z0-9]* deaths\y/, a)) 
     split(a[0],b,":|deaths")
     return b[2]
 }
 return -1
}

There are other ways to do it via the command line, but I need inside a function in a script using readfile().

The above code returns the correct birth year, but the death year is mangled because the regex is grabbing both the birth and death strings.

It works when not using the readfile() function, instead getline and this regex

match(article,"Category:[A-z0-9].* deaths", a)

The ".*" grabs everything to the end of the line and since readline makes the entire file a single line it grabs to the end of the "line" (file). That's why I'm using word boundary ("\y"), which works, but it doesn't work if there is a space in the data, such as the case here with the death string ("2nd C."). I tried adding "[:space:]" but that didn't work. I think this is solvable with the right regex but I'm out of ideas.

2

Why does it have to use readfile()? Why not use awk the way it was meant to be used? Is this homework?

3

OK - I found the problem. The match line should look like this:

match(article,/\yCategory:[[:space:].A-z0-9]*deaths\y/,a)

The problem was [: punct:] was matching the [[]] characters in file.txt .. so in order to match the "." in "2nd C." it's now noted directly (right before the A-z).

Thanks.

4

The RE [A-z] also includes the characters [ , \ , ] , ^ , _ , and ` . The RE [[:space:]] contains all whitespace characters; I'm guessing that you just want a space character instead. And, if you're trying to catch common forms of dates, you probably also want to include comma (for dates like December 25, 1999 . So, a better RE would probably be:

match(article,/\yCategory:[ .,[:alnum:]]* deaths\y/,a)
5

Note: A-z0-9 will probably not do what you want:

$ echo \[ | grep '[A-z0-9]' 
[

This is because square brackets fall within that range. Moreover, ranges like that are also dependent on locale which could produce other unexpected results. So it would be better to use [:alnum:] instead.

---
Also the code looks a bit convoluted for such a simple task. I don't see why you would need to use gawk and read the entire file in memory, while this could also be done by using awk's line processing mid section, which is typically used for this. I would suggest you read up on that.

---
You could perhaps also consider selecting a different line processing tool like GNU sed

sed -rn 's/.*\[\[Category:(.*) (births|deaths)\]\].*/\u\2: \1/p' file

Which would maybe produce similarly acceptable results..

Births: 98
Deaths: 2nd C.
6

Thank you, Don. That is much better. I did not realize A-z was catching the square brackets which appears to be the underlying problem.

Scrutinizer, it's part of a larger Awk script which passes the "fp" variable around to different functions for processing, not a shell script.