Removal of HTML ASCII Codes from file

Shell Programming and Scripting
1

Hi all,

I have a file with extended ASCII codes in the description which needs to be removed.

List of extended ascii codes

"�", "�", "�", "�", "�", "�", "-", "-", "�",
"'", "�", "�", "�", "�","�", "�", "�",
"...", "�", "�", "�"

Sample data:

Test Details-HAVE BEEN PUBLISHED on date 8/11
Please tag�the notes and activate the pool'in all systems and reporting programs by 01/21
Select�new TRIP pool for sale.�Pick letter to be sent 6/26
Test - Obama w/ Sys.Admin�rights
Description would go here�Clinton!
files documents by wikileaks�search

Using sed to remove the codes

sed -e 's/[�]*[�]*[-]*[']*[�]*[�]*[�]*[�]/ /g' spcl_char.dat > spcl_char_2.dat

Test Details HAVE BEEN PUBLISHED on date 8/11
Please tag the notes and activate the pool in all systems and reporting programs by 01/ 1
Select new TRIP pool for sale. Pick letter to be sent 6/ 6
Test - Obama w/ Sys.Admin rights
Description would go here Clinton!
files documents by wikileaks search

The output has any occurrence of the number present in the description removed, however 26 has been converted to 6 and 21 to 1.

How can I strip/remove the extended ascii codes without the description being touched.

Thanks.

2

You will find out that some special characters are very hard to delete.

Thus, you can work by reversing the logic and keeping the characters you do not want:

sed 's/[^a-zA-Z0-9]/ /g' File

Keep adding more characters inside of the brackets as you need them.