Identifying a sentence and putting it on a new line

Shell Programming and Scripting
1

I am revisiting the problem of sentence splitting. I have a Perl Script which splits a para into sentences, but acronyms and short forms create an issue

#!/usr/bin/perl

use feature qw/say/;
use strict;
use warnings;

my $s;
my @arr;

while(<>) {
  chomp $_;
  $s .= $_ . " ";
}
@arr = $s =~ m/[A-Z].+?[.;](?=[^.;][A-Z]|\s*$)/g;
foreach (@arr) {
    say;
}

I have identified a list of abbreviations from a large corpus (not necessarily exhaustive) which are given below and which I would like to integrate in the script but since I am still learning Perl, I have not been able to integrate them. I am giving below the list of such cases. The list is not complete and can be added to. The syntax is as under:

Abbr["followed by the abbreviation];

Abbr["Co."];
Abbr["Corp."];
Abbr["vs."];
Abbr["e.g."];
Abbr["etc."];
Abbr["ex."];
Abbr["cf."];
Abbr["eg."];
Abbr["Jan."];
Abbr["Feb."];
Abbr["Mar."];
Abbr["Apr."];
Abbr["Jun."];
Abbr["Jul."];
Abbr["Aug."];
Abbr["Sep."];
Abbr["Sept."];
Abbr["Oct."];
Abbr["Nov."];
Abbr["Dec."];
Abbr["jan."];
Abbr["feb."];
Abbr["mar."];
Abbr["apr."];
Abbr["jun."];
Abbr["jul."];
Abbr["aug."];
Abbr["sep."];
Abbr["sept."];
Abbr["oct."];
Abbr["nov."];
Abbr["dec."];
Abbr["ed."];
Abbr["eds."];
Abbr["repr."];
Abbr["trans."];
Abbr["vol."];
Abbr["vols."];
Abbr["rev."];
Abbr["est."];
Abbr["b."];
Abbr["m."];
Abbr["bur."];
Abbr["d."];
Abbr["r."];
Abbr["M."];
Abbr["Dept."];
Abbr["Mr."];
Abbr["Jr."];
Abbr["Ms."];
Abbr["Mrs."];
Abbr["Dr."];

How do I integrate these and ensure that when the script encounters the above exceptions, it does not treat the full-stop as a sentence delimiter as in in the examples below?

Mr. Smith said today is Jun. 15th.
Jones Inc. filed for bankruptcy.

A couple of examples for such integration would suffice and I will integrate the rest.
Many thanks

2

Hi.

Perhaps these cpan modules may help;I have not used the first, but I have used the second with success In addition to the links below, see post #9 in thread Using awk to find sentences. for for a demonstration ... cheers, drl

Above from:

Above from:

3

How about this awk solution. function sentence_parse will populate an array with each sentence.

Features:

  • ignores acronyms from the supplied file
  • ignores any text within quoted strings
  • removes leading spaces (spaces following previous full-stop)
awk '
function rindex(str,c)
{
  return match(str,"\\" c "[^" c "]*$")? RSTART : 0
}

function sentence_parse(text,sentences,quote,start,num,pos,last_word) {
   quote = (quote ? quote : "\"")
   start=num=1
   for(pos = 1; pos <= length(text) ; pos++) {
       char=substr(text, pos,1)
       if(start && char != " ") start=0
       if(!start) {
           sentences[num] = sentences[num] char
           if(char==quote) inquote=!inquote
           if (!inquote && char == ".") {
              last_word = substr(substr(text,1,pos), 1+rindex(substr(text,1,pos), " "))
              if(!(last_word in acro)) {
                  start=1
                  num++
              }
           }
        }
    }
}

NR==FNR{
 if($0 ~ "^Abbr\\[\"") {
     split($0,vals,"\"");
     acro[vals[2]]
 }
 next
}

{ 
   delete sen
   sentence_parse($0, sen)
   for(i=1; i<=length(sen) ; i++) print sen
} ' acronyms.txt infile

Credit to vgersh99 for the rindex function