HOw to find special characters

sol_nov · February 16, 2012, 5:28pm

I have flat file which has data like this

glid�as_liste�025175456

How can I print these lines into new file?

methyl · February 16, 2012, 5:46pm

Sorry, your question does not make sense.
Please post sample input, expected output and an explanation of the process.
Please also post what Operating System and version you have and what Shell or programming language you prefer.

The upside-down quesion mark usually denotes a character which your display equipment cannot display. Do you know what character(s) these are?

Scrutinizer · February 16, 2012, 5:46pm

� Or do you mean only lines that contains � ?

grep � file > newfile

chihung · February 16, 2012, 8:07pm

"od -c" can help you identify those non-printable character and represent them in octal. If you want to only print those lines:

while read line
do
  echo "$line" | od -c | grep -w -e '[0-7][0-7][0-7]' > /dev/null && echo "$line"
done < your-input-file

alister · February 16, 2012, 10:25pm

chihung:

"od -c" can help you identify those non-printable character and represent them in octal. If you want to only print those lines:
while read line
do
  echo "$line" | od -c | grep -w -e '[0-7][0-7][0-7]' > /dev/null && echo "$line"
done < your-input-file

That echo statement cannot be relied upon. What if $line expands to something that looks like a command option? You can't use "--" to explicitly signal the end of option processing. printf %s "$line" is a better approach.

Perhaps it would also be a good idea to suppress od's offset with -An, since the offset's format is unspecified and could possibly be matched by grep.

AWK might be easiest to use since its regular expression flavor is required to support octal escape sequences even within bracket expressions. However, without a definition of "special characters" by the OP, that's just speculation.

Regards,
Alister