sed error: invalid reference

Shell Programming and Scripting
1

Hello all,
I am using sed to parse a particular part of a string and am having problems. I am getting the following error:
sed: -e expression #1, char 28: invalid reference \1 on `s' command's RHS
Here is the code I am using:

echo "Alarm SET:"
echo ""
echo "Date:            " $DATE
echo "Time:            " $TIME
echo "ModelName:       " $MNAME
TST=$(echo "$EVENTMSG" | sed -ne 's/.*odelName=(\w+[^.]+)/\1/p')
echo "Non-FQDN Name:    " $TST
Severity:                  " $SEV

Here is the output:

Alarm SET:
Date:             10/01/2012
Time:             13:26:49
ModelName:        twa-casql01.c.com
sed: -e expression #1, char 28: invalid reference \1 on `s' command's RHS
Non-FQDN Name:
Severity:         MAJOR

As you can see, something is not working when I try to parse $EVENTMSG. My expected outcome is:

Alarm SET:
Date:             10/01/2012
Time:             13:26:49
ModelName:        twa-casql01.c.com
Non-FQDN Name: twa-casql01
Severity:         MAJOR

Ultimately, I am trying to verify that the $MNAME is the non-FQDN formated name as the ModelName contained in the $EVENTMSG.

The regex syntax of:

(\w+[^.]+)

takes the FQDN (or Fully Qualified Domain Name) and pulls the dot (.) suffix off of it.

Any ideas?

MANY THANKS IN ADVANCE!!

2 
echo "Alarm SET:"
echo ""
echo "Date:            " $DATE
echo "Time:            " $TIME
echo "ModelName:       " $MNAME
echo "Non-FQDN Name:   " ${EVENTMSG%%[.]*}
echo "Severity:        " $SEV
1 Like
3

As usual in your postings, you leave out a lot of details that are needed to help solve your problem:

  1. What is in the shell variable EVENTMSG? For the purposes of this thread, I will assume that it contains the lines:
    text ModelName= twa-casql01.c.com
    and
    text AlarmSeverity= Critical
    and several other lines before and/or after these lines.
    On these lines, I have no idea whether the whitespace between ...= and the following data is one or more <tab>s, one or more <space>s, or a combination of <space>s and <tab>s. The search expressions used below allow any combination of one or more <space> and/or <tab> characters.
  2. You don't say what system you're using. It is obvious why you're getting the error you're getting on your sed command (you have a request to use backreference #1 in your replacement string, but you didn't define a subexpression in your BRE), but it isn't obvious whether or not the \w would be treated as a match for whitespace (as in perl) or treated as an escaped w as specified by the standards for BREs in the sed utility. And, BREs in the standard don't treat + as special in a BRE. (It means match one or more of the previous expression in an ERE, but it just matches a <plus-sign> in a BRE.
  3. You search for odelName= , but you seem to have ModelName: in what you want output. I will assume for now that $EVENTMSG contains <equals-sign>s, but you want to translate them to <colon>s in your output.

The suggested text here should work on UNIX-, Linux-, and BSD-systems as long as you're using a POSIX compatible shell (such as bash or ksh). (I use ksh so it is what is specified here, but I'm not using any ksh specific features.)

This example is an amalgam of your original posting and the updated text in your original posting:

#!/bin/ksh
EVENTMSG='Wed 26 Sep, 2012
 ModelType= Event message testing
 ModelName=        twa-casql01.c.com
 AlarmSeverity= Critical
 ActionEvent= Evacuate Immediately
 Payload= large raw data string'

SEV=$(printf "%s\n" "$EVENTMSG" | sed -ne "s/.*AlarmSeverity=[  ]*\(.*\)/\1/p")
TST=$(printf "%s\n" "$EVENTMSG" | sed -ne 's/.*ModelName=[      ]*\([^.]\{1,\}\).*/\1/p')

printf " AlarmSeverity:\t%s\n Non-FQDN Name:\t%s\n" "$SEV" "$TST"

Note that the first bracet expression in both sed statements is the four character sequence: <left-square-bracket>, <space>, <tab>, and <right-square-bracket>.

This script produces the following output on OS X (and should be the same on any system with a sed conforming to the standards:

 AlarmSeverity:	Critical
 Non-FQDN Name:	twa-casql01

I hope this helps,
Don

1 Like
4

First and foremost, thank you for all the time you are putting into helping me! I am not the best scripter (I know, it's quite obvious)!!

Here is the code I used. Note, I am including everything in the shell script that has to do with my output:

echo_info()
{
echo " "
echo "============>>>    NEW Alarm Notification from SPECTRUM    <<<============"
echo "=========================================================================="
echo "      A ${SEV} ALARM JUST OCCURRED ON ${MNAME}"
echo "=========================================================================="
echo ""
echo "===>>>  ALERT TYPE: ${ONELINE}"
echo ""
echo "Date:                 " $DATE
echo "Time:                 " $TIME
echo "Alarm Severity:  " $SEV
echo "Alarm Acknowledged? " ${ACK}
echo " "
echo "###########  DEVICE/MODEL INFORMATION  ###########"
echo "${MNAME} is modeled on ${SERVER} "
echo "IPAddress:       " $IPADDRESS
echo "ModelName:       " $MNAME
echo ""
echo "Global AlarmID:  " $GLOBAL_ALARM_ID
#******************************
#        Test Section
#******************************
echo ""
DJL=$(printf "%s\n" "$EVENTMSG" | sed -ne "s/.*Device\s[  ]*\(.*\)/\1/p")
ABC=$(printf "%s\n" "$EVENTMSG" | sed -ne 's/.*of\stype\s[      ]*\([^.]\{1,\}\).*/\1/p')
printf " MODEL NAME:\t%s\n MODEL TYPE:\t%s\n" "$DJL" "$ABC"
 
###########################################################################
#  The following parameters contain values only when
#  the SPECTRUM Alarm Notification Manager is enabled.
###########################################################################
if [ "$SANM" ]
then
echo ""
echo "SANM AlarmAge Setting: ${AGE} minute(s)"
echo "NotificationData:" $NOTIFDATA
echo ""                                         # insert blank line
fi
# This variable has substituted placeholder chars, echo them inside quotes
echo "EventMessage:     $EVENTMSG"
echo ""
echo ""
}

When an alarm came in that followed the alarm filtering criteria the output is as follows:

============>>>    NEW Alarm Notification from SPECTRUM    <<<============
==========================================================================
      A MAJOR ALARM JUST OCCURRED ON lp-webtactst01
==========================================================================
 
===>>>  ALERT TYPE: MANAGEMENT AGENT LOST
 
Date:                  10/03/2012
Time:                  08:58:02
Alarm Severity:   MAJOR
Alarm Acknowledged?  No
 
###########  DEVICE/MODEL INFORMATION  ###########
lp-webtactst01 is modeled on uta-specpol01 
IPAddress:        10.10.15.250
ModelName:        lp-webtactst01
ModelHandle:      0x206dd0
DeviceType:       Net-SNMP Linux
Mtype:            Host_Device
ModelTypeHandle:  0x1160089
 
Global AlarmID:   506c527a-bfe8-1006-02cd-0050569d7aa8
 
 MODEL NAME:    is no longer responding to primary management requests (e.g. SNMP), but appears to be responsive to other communication protocol (e.g. ICMP).  This condition has persisted for an extended amount of time.  An alarm will be generated.   (event [0x00010daa])
 MODEL TYPE:     Host_Device is no longer responding to primary management requests (e
 
SANM AlarmAge Setting: 0 minute(s)
NotificationData:
 
EventMessage:     Wed 03 Oct, 2012 - 08:58:02 - Device lp-webtactst01.c.utah.com of type Host_Device is no longer responding to primary management requests (e.g. SNMP), but appears to be responsive to other communication protocol (e.g. ICMP).  This condition has persisted for an extended amount of time.  An alarm will be generated.   (event [0x00010daa])

As you can see, I am not getting my desired outcome. I was expecting the following output:

============>>>    NEW Alarm Notification from SPECTRUM    <<<============
==========================================================================
      A MAJOR ALARM JUST OCCURRED ON lp-webtactst01
==========================================================================
 
===>>>  ALERT TYPE: MANAGEMENT AGENT LOST
 
Date:                  10/03/2012
Time:                  08:58:02
Alarm Severity:   MAJOR
Alarm Acknowledged?  No
 
###########  DEVICE/MODEL INFORMATION  ###########
lp-webtactst01 is modeled on uta-specpol01 
IPAddress:        10.10.15.250
ModelName:        lp-webtactst01
ModelHandle:      0x206dd0
DeviceType:       Net-SNMP Linux
Mtype:            Host_Device
ModelTypeHandle:  0x1160089
 
Global AlarmID:   506c527a-bfe8-1006-02cd-0050569d7aa8
 
 MODEL NAME:    lp-webtactst01.c.utah.com
 MODEL TYPE:     Host_Device
 
SANM AlarmAge Setting: 0 minute(s)
NotificationData:
 
EventMessage:     Wed 03 Oct, 2012 - 08:58:02 - Device lp-webtactst01.c.utah.com of type Host_Device is no longer responding to primary management requests (e.g. SNMP), but appears to be responsive to other communication protocol (e.g. ICMP).  This condition has persisted for an extended amount of time.  An alarm will be generated.   (event [0x00010daa])

What am I doing wrong?:confused::confused:

5

In message #2 in the thread titled Parsing a $VARIABLE within a script
in the forum titled Shell Programming and Scripting, you said:

but in the last message you show that $EVENTMSG is Wed 03 Oct, 2012 - 08:58:02 - Device lp-webtactst01.c.utah.com of type Host_Device is no longer responding to primary management requests (e.g. SNMP), but appears to be responsive to other communication protocol (e.g. ICMP). This condition has persisted for an extended amount of time. An alarm will be generated. (event [0x00010daa]) which, of course, contains neither "ModelType=" nor "ModelName=". So, I have to conclude that despite the fact that all of the threads you have started in this forum seem related, I can't rely on statements made in one thread being of any help in any other thread. Nonetheless, this and other comments about event message formats in other threads are the only specifications I have for the format of an event message.

So, back to the issues at hand. In the first message in this thread you said that the commands:

TST=$(echo "$EVENTMSG" | sed -ne 's/.*odelName=(\w+[^.]+)/\1/p')
echo "Non-FQDN Name:    " $TST

gave you the diagnostic message:

sed: -e expression #1, char 28: invalid reference \1 on `s' command's RHS

when you were expecting the output:

Non-FQDN Name: twa-casql01

As noted earlier, the diagnostic message came about because the replacement in your sed substitute command referenced subexpression 1 in the search pattern, but you didn't have any subexpressions in your search pattern. Now that you have actually shown us in message #4 in this thread what is in $EVENTMSG, we also now know that the string being searched doesn't contain the requested pattern (i.e., "odelName" doesn't appear anywhere in $EVENTMSG). To get the output you want from the value of $EVENTMSG you have supplied, the following commands will work:

TST=$(echo "$EVENTMSG" | sed -ne 's/.* Device \([^. ]*\).*/\1/p')
echo "Non-FQDN Name:     $TST"

or, without using sed:

TST=${EVENTMSG#* Device }
TST=${TST%%[. ]*}
echo "Non-FQDN Name:     $TST"

Note that the bracket expression identifying the end of the Device name is looking for <period> or <space> to end the matched name. This is because in some of your earlier posts you said that sometimes the name contained one or more <period>s (and, if so, you wanted them to be removed) and sometimes did not contain a <period>. Either of the above methods of setting $TST will terminate the string to be returned at the first <period> or <space>.

I hope this resolves this problem for you,
Don

1 Like