SED: Replacing $1 with $2 escape problem

Shell Programming and Scripting
1

Hi all,

I have a script which uses sed to replace one string with another. The problem is, the string to be matched, and its replacement are coming in as two command line arguments $1 and $2

$1 and $2 can be absolutely anything, but both should be treated purely as strings. My sed command below doesn't do any escaping which is what I need.

sed "s/$1/$2/g" /tmp/input.txt > /tmp/output.txt

I spent a couple of days making some code which could escape these variables before use in my sed command and it seemed to allow me to enter almost anything but today, when trying to replace one string, with another string that contains an ampersand, some fancy sed backtracking came into play.

function escape {
new=`echo "$1" | sed 's/\\\\/\\\\\\\\/g' | sed 's/\//\\\\\//g' | sed 's/\*/\\\\\*/g' | sed 's/"/\"/g' | sed 's/\"/\\\\\"/g'`
echo $new
}

What I think I really need is a premade escape function for use in sed, or maybe some arguments to sed itself to ignore all special chars and treat them as strings.

2

Give some examples of lines of your input file, the string to be matched, its replacement and the desired output.

Regards

3

Hi,

if you want to escape a string, simply use printf with "%q":

var_a=$(printf "%q" $1)
var_b=$(printf "%q" $2)

sed "s/${var_a}/${var_b}/g" file

HTH Chris

4

Thanks for the printf command, I never knew that could do shell escaping for me.

That seems to escape most of the characters that could break my sed, but still if I use an ampersand in my second string, sed blows up on me somehow.

An example I am using to try this out is

Replace "abcd" with "\Bells & Whistles/"

So print f shows "\Bells & Whistles/" as
"\\Bells\ \&\ Whistles/" when var printed to screen
"Bells\ \&\ Whistles/" when var echoed to file

Two problems with this for sed.

1) The special character in sed "&" still seems to be picked up by sed as a special character.

2) The special character in sed "/" doesn't get escaped by printf (should be easy to get around)

Its a pity sed doesn't let me get around the problems I'm having with these special characters. I'm imagining if I manage to fix these special char problems with your help, that down the line I will find more characters that break it....

5

Hi, I too am new to Unix but discussion my earlier question might be of ome help to you.
View Discussion

6

Thanks for the link.

I actually fixed the problem with the forward slash. Now its just a matter of fixing the problem with the ampersand being treated as a sed special char.

Here is the code so far

function sed_escape {
new=`echo "$1" | sed 's/\//\\\\\//g'`
echo $new
}
VAR1="unimportant"
VAR2=$(printf "%q" "$1") // does most of the work for me
VAR2=`sed_escape "$VAR2" // takes care of forwardslash character and hopefully with your helpall sed specials like &

sed "s/${VAR1}/${VAR2}/g" // my actual sed command

So just the ampersand is causing me troubles now, im sure other sed specials like ^ will be a problem once the ampersand is fixed.

7

Oh no, don't do that. You need no function at all. Use

s#var_a#var_b#g

or every other character you like which you will not likely encouter
in the string instead.

With gnu linux and sed there is no problem at all:

c="/bells/ & /whistles/"    
d=$(printf "%q" $c)         
sed "s#bells#${d}#" <<< $c

And will give you:

//bells/ & /whistles// & /whistles/

But as Franklin52 pointed it would be the best to give us some examples
of what you have and what you want.

HTH Chris

8

Thanks Christoph.

The problem is, the string to be replaced will be given by a user on a webpage which allows the user total freedom.

This is then used as input to sed, which replaces a line in a word document. The user expects any character on the keyboard they enter, to be displayed flawlessly in the word doc, so its not a case of choosing characters which may not be used, as any character can be used.

So if I choose / as the delimitor, and escape that out manually, your saying I should be ok to go?

Its strange how & doesn't cause you troubles and it does for me, I will investigate this further, but I don't think printf's escaping, will escape all sed specials for me.