How can I get rid of the ` character from input file?

Shell Programming and Scripting
1

A sed command works most of the time however it fails sometimes.

I put each line (record) I read of a file through the following command
data=$(cat file | sed 's/[^a-zA-Z0-9+_:-]//g' | sed 's|.*ex:Msg\(.*\)ex:Msg.*|\1|' )

When I run the script I get a message that states that there is an invalid format character.

The list of invalid chracters I am getting is as below, is there a way to solve this issue?. How can I get rid of the ` character from the input data?

`w'
`m'
`k'
``'
`;'
`O'
`m'
`H'
`^'
` '
`:'
`v'
`S'
`k'
`j'
`)'
`!'
`['
`m'
`@'
`T'

2

Since the character is the first in each line...

cat file | cut -c2-
3

I am afraid its not the first character, I have specified the message that is displayed on the screen when I run my script which is of the format ./records.sh invalid chracters....etc

4

Can you provide a sample of the input file

5

To answer the question in the subject line:

tr -d '`' < FILE

Under what conditions does it fail?

I hope you don't really run the entire file through two instances of sed (not to mention cat) for every line in the file.

You are using three external commands where you only need one:

data=$( sed -e 's/[^a-zA-Z0-9+_:-]//g' -e 's|.*ex:Msg\(.*\)ex:Msg.*|\1|' file )

Please post the exact message you get. (Cut and paste it, don't retype it.)

Why do you think the problem is the ` character?

The invalid characters are those between the quotes.

6

cfajohnson, unfortunately I cannot post a copy of the data because I am not allowed, confidential info. The only reason I know that these chars are causing a problem is because I get an on line message: ./counter.sh line 5: printf: `w: invalid format and this is repeated for characters that I have mentioned in my initial posting. The contents of counter.sh is:

counter=1
while read line
do
# Test the file
printf "$line" > temp$counter
pref=$(sed <$temp$counter -e 's/[^a-zA-Z0-9+_:-]//g' -e 's|.*ex:Msg\(.*\)ex:Msg.*|\1|')
printf"
let counter=counter+1
done < temp01

Also you mention that I could use only one command instead of three for extarcting the data. Could you please specify? I am new to scripting so any help would be greatly appreciated.

Code:
data=$( sed -e 's/[^a-zA-Z0-9+_:-]//g' -e 's|.*ex:Msg\(.*\)ex:Msg.*|\1|' file )

7

As the message says, you are using an invalid format character with printf.

The syntaxt for printf is:

printf FORMAT_STRING ARG ...

What you need is:

printf "%s\n" "$line"

Please put code inside code tags.

Don't use sed for every line in a file; it will be extremely slow.

Use sed on the entire file and work on its output.

Use the standard form for arithmetic:

counter=$(( $counter + 1 ))