Split email message with procmail

Shell Programming and Scripting
1

Hi,

I'm using procmail to ingest incoming messages. The goal is to split the message body into chunks and forward those to a different adddress. Here's a sample of the message body:

Group: testgroup
---------------------------------------
Host: test1.local:30200  
Type: conf
Alert: Host Down
Status: Open
Last Updated: 2013/06/13 14:11 GMT
Created: 2013/06/13 14:11 GMT
---------------------------------------
Host: test1.local:30000  
Type: conf
Alert: Host Down
Status: Open
Last Updated: 2013/06/13 14:11 GMT
Created: 2013/06/13 14:11 GMT
---------------------------------------
Host: test1.local:30500  
Type: conf
Alert: Host Down
Status: Open
Last Updated: 2013/06/13 14:11 GMT
Created: 2013/06/13 14:11 GMT
---------------------------------------
---------------------------------------
Configure alerts: Go to this URL

The trick is to extract anything between the 2 delimiters (/^Host:/ && /^Created:/) and mail them. The sample above should result in 3 distinct messages:

Message1:

Host: test1.local:30200  
Type: conf
Alert: Host Down
Status: Open
Last Updated: 2013/06/13 14:11 GMT
Created: 2013/06/13 14:11 GMT

Message2:

Host: test1.local:30000  
Type: conf
Alert: Host Down
Status: Open
Last Updated: 2013/06/13 14:11 GMT
Created: 2013/06/13 14:11 GMT

Message3:

Host: test1.local:30500  
Type: conf
Alert: Host Down
Status: Open
Last Updated: 2013/06/13 14:11 GMT
Created: 2013/06/13 14:11 GMT

Thanks for the help!

Waltari

2

Is the destination address and subject in there? A simple sed script can turn it into a shell script that mails with here files.

mailx -s something-from-inside something-else@some_mail_server <<!
Host: test1.local:30500  
Type: conf
Alert: Host Down
Status: Open
Last Updated: 2013/06/13 14:11 GMT
Created: 2013/06/13 14:11 GMT
!
3

Actually the Subject header is dynamic and will contain the concatenation of the
values of the Alert, Host & Created lines (e.g. Host Down on test1.local:30000 at 2013/06/13 14:11 GMT).

I've managed to use awk as follows:

cat msg.file | awk '/^Host:/,/^Created:/ { print $0 }'

But I need to process each extract between my two delimiters.

Thanks!

W

4

Well, awk and sed can do that: awk can collect data in variables for later printing, and sed can get the whole block in the buffer using N. then pick out values with an s to build the shell wrapper with mail command.

We need requirements that define the subject and email address.

5

Thanks DGPickett for you input!

The format of the Subject line is described above. The Sender (From) address will be the hostname portion of the value of the "Host:" line (e.g. admin@test1.local) and the Recipient is static (postmaster@mydomain.com).

W

6

You want to forge a remote sender? I just wanted a recipient and subject.

In my experience, a time stamp somewhere in the subject is nice, for selecting email by subject, perhaps for sorting, too, "<General_topic>-Hostname-YYYY-MM-DD_HH:MM".

So, in awkish, you need to collect all lines of the block in a body variable and collect bits for the subject and recipient in two or more variables, and the after collecting the Created line, spit out the mail command and input data and maybe clean your variables. Or you can clear your variables on the first line of the block. If your recipient variable ends up empty, you might want to send to a default address with a 'DefectiveBlock:' prefixed subject.

7

The outgoing messages resulting from the split will be consumed downstream by our monitoring system, which will convert the email message (headers + body) into an event.

Since the incoming message could contain alerts on multiple hosts, I need to break it down into individual messages, one for each 'host'.

I'm already including a time stamp in the subject line (derived from the Created line).

Does this make sense?

8

If there are many blocks going to the same place, then it would be nice to concatenate them in a file. Since awk can rearrange stuff, one line per event might be nicer, pipe separated or CSV. Can awk write to data-constructed file names?

Once awk builds files for each destination in an outgoing directory, a shell layer can send each file based on the file name.

9

I've managed to do that with awk:

/^Host:/{
  split($2, h, ":");
      subject = "[" $2 "]";
  command = "mail -r test@" h[1];
  a = $0;
  next
}
/^Alert:/{
  split($0, h, ": ");
      subject = subject FS "[" h[2] "]"
}
{
  a = a RS $0
}
/^Created:/{
  subject = subject " on [" $2 " " $3 " "$4 "]";
  command = command " -s \"" subject "\" admin@myhost.domain"
  print a | command;
  close(command);
}
10

Sometimes I think it would be simpler to connect to localhost:25 and just send them with (e)smtp than use mail or sendmail! Of course, then you are connected to sendmail, but the options are open and the rules are pretty well understood. :smiley:

A bonus would be to send them direct to the end host port 25, and skip local email overhead.