Trouble with sed and substituting a string with special characters in variable

Hey guys,
I know that title is a mouthful - I'll try to better explain my struggles a little better...

What I'm trying to do is:

1. Query a db and output to a file, a list of column data.
2. Then, for each line in this file, repeat these values but wrap them with:

ITEM{
          LABEL = <LINEVALUEFROMFILE>
          VALUE = <LINEVALUEFROMFILE>
}

for each line in the file. I'm doing this and outputting that to a new file, and purging the old.
3. CAT the contents of the formatted output file, then substitute it into a .cfg file in place of a "Marker" or string value that is unique (in my case, a string called VENDORLIST.

I have everything working fine right up to the sed statement. I've tried using:

sed -i "s/$marker/$markerrepl/g" $formattedfile

but that returns:

sed: -e expression #1, char 60: unterminated `s' command

I've tried "protecting" the variables by single quoting them like this:

sed -i "s/'$marker'/'$markerrepl'/g" $formattedfile

but that returns a similiar error with a different character reported, and if I wrap the variables in "" it merely substitutes the marker string with the variable in double quotes (not the content of the variable).

Speaking of the variable content, at the end of the script I echo it out to screen so I know the content is there... all I can figure is there is an issue with characters within that variable and that's the problem...

By the way, here's an example of what that variable content looks like from the formatted file (step 2):

 
                                          ITEM{
                                                          LABEL = Panasonic Corporation of N.A.
                                                          VALUE = Panasonic Corporation of N.A.
                                          }
                                          ITEM{
                                                          LABEL = NewWave Technologies, Inc.
                                                          VALUE = NewWave Technologies, Inc.
                                          }
                                          ITEM{
                                                          LABEL = Kyocera Doc Solutions of America, Inc.
                                                          VALUE = Kyocera Doc Solutions of America, Inc.
                                          }
                                          ITEM{
                                                          LABEL = Ingram Micro, Inc.
                                                          VALUE = Ingram Micro, Inc.
                                          }

Are there any sed or awk experts out there can help me get this data substituted in for my marker string?

You have told us everything - save for the contents of "$marker" and "$markerrepl", which would be the only thing necessary to know. How am i - how are we - supposed to give you advice regarding a string when we don't know the string?

I hope you aren't too dissatisfied with an answer which will be of only general nature:

When substituting from variable contents there are a few problems. The first is that this is prone to "injections" - a part of the search- or replacement-string will constitue a sed command itself. The output will probably not be what you had in mind. Example:

a="abc"
b="xyz"

sed "s/$a/$b/" /some/file       # will work as expected

a="abc/xyz"
sed "s/$a/$b/" /some/file       # will not work

A solution for this will be to escape all the characters special to sed prior to useing these strings. Escaping in sed is done with backslashes:

a="abc\/xyz"
b="xyz"

sed "s/$a/$b/" /some/file       # will work as expected

A second problem is shell-related: without quoting variables can break up lines into pieces. This can be partly overcome with proper quoting:

a="a b c"
b="x y"

sed s/$a/$b/ /some/file            # will not work
sed 's/'"$a"'/'"$b"'/g' /some/file # will work

This will not help you if variables contain line feeds.

I hope this helps.

bakunin

Thanks for your reply, bakunin. Sorry it wasn't more clear, but in the OP, I was trying to convey that $marker was the string itself that I wanted to replace (VENDORLIST), and $markerrepl was the cat(ed) results of the formatted file I gave an example of:

                                                          ITEM{
                                                          LABEL = Panasonic Corporation of N.A.
                                                          VALUE = Panasonic Corporation of N.A.
                                                          }
                                                           ITEM{
                                                          LABEL = NewWave Technologies, Inc.
                                                          VALUE = NewWave Technologies, Inc.
                                                          }
                                                           ITEM{
                                                          LABEL = Kyocera Doc Solutions of America, Inc.
                                                          VALUE = Kyocera Doc Solutions of America, Inc.
                                                          }
                                                          ITEM{
                                                          LABEL = Ingram Micro, Inc.
                                                          VALUE = Ingram Micro, Inc.
                                                          }

Anyway, I guess we can close this. I gave up on sed and trying to figure out how to escape the special characters and just used perl to accomplish the desired result.

Thanks again for your help!

Ok, sorry for not getting this earlier. So you search for a certain string and want to replace this string with the content of a file. This is relatively easy to do with the "r <file>" command, which reads a file.

As you said you already have a solution in PERL i won't go into details here, because it is non-trivial to do: You first have to split the line at the word searched for, put the part after it to hold space, delete the word itself, then read the file and append the content of the hold space back after. Finally put this whole procedure into a loop to cover for multiple occurrences of the search term.

bakunin