sed command not working inside ksh script but works fine outside

Shell Programming and Scripting
1

Hi,

I am a bit confused ,why would a sed command work fine outside of ksh script but not inside.

e.g

I want to replace all the characters which end with a value and have space at end of it.
so my command for it is :

sed -i "s/$SEPARATOR /$SEPARATOR/g" file_name

This is working fine in and outside of the ksh script.

Another e.g

sed -i "s/$SEPARATOR$//g" file_name

I want to remove last character from every line in the file.So the above command works fine outside of ksh script but not inside.
Also,

I am trying to execute below command to remove all Acaret junk characters.

sed -i "s/�//g" file_name

This too works fine outside of ksh script but not inside.

Could somebody please help me explain why is it so and what can I do to resolve this.Also,what difference is it if I use double quotes in sed instead of single quotes.

Please help.:confused:

2

I can't see a reason why those sed commands should not work within scripts. Is the SEPARATOR variable assigned to correctly? Execute the scripts with the -x option set, and post the log as well as input and output files (drop the -i for this).

1 Like
3

Hi,

Thanks for the suggestion I checked with the -x flag and found that the value is not resolved for $SEPARATOR.
Still I am facing issues with the caret A £ symbol from the string.I am trying to remove it from the file completely so using the below command:

sed -i "s/�//g" file_name

But it does not make any changes to the file and the £ is still seen.
But this above command when submitted from outside the script works.
Any way to remove this from the file ?

4

Hi,
For the first command sed that not work in ksh script, try as:

sed -i "s/${SEPARATOR}\$//g" file_name

But, maybe not work again, what is $SEPARATOR ?

How to launch your ksh script ?

For information, accented character may not be in sed. these depend your locale.

EDIT: try as:
1- how to find octal value of character:
In my example, my text in File is FOO�BAR.

$ cat File
FOO�BAR
$ sed -n 'l160' file
FOO\303\240BAR$

Here, I see that character "�" is in octal \303\240 but really value is \0303\0240.
And to replace "�" by nothing:

$ sed  $(echo -e 's/\0303\0240//g') File
FOOBAR

Regards.

5

@disedorgue: That's not quite right. � is a two (or more) byte unicode character; sed will find it and replace it (at least my GNU sed version 4.2.1 does).
@vital_parsley: As long as you don't post the info requested, I'll be incapable to help.

6

Thanks guys for all the help.I followed your responses and found that my first mistake was the $SEPARATOR variable was out of scope of the block so was not resolving correctly.
Secondly the caret A was not getting removed from the file is because ..when the job ran the wrapper script changed the LANG variable and so the caret A was not getting removed.I have explicitly exported it in the script and now its fine.

Thanks guys for all the help!

7

@RudiC:Yes and No, sed will found in locale use.
If I use charmap ISO-8859-1:

$ echo "�" | sed -n 'l'
\302$

If I use charmap UTF-8:

$ echo "�" | sed -n 'l'
\303\202$

And therefore sed might not found character in file because it is not the same character set.

Regards.

8

Yes, you are right. But, if the file prints OK with one charset, and you don't change locale in between, the sed expression should work as well.

9

I agree with this but it also depends on which is written the script ...