Copying a file with UTF char on UNIX server

Shell Programming and Scripting
1

Hi,

I need to run a SQL which check for special UTF char in DB. When I try to copy that in UNIX file it changes it to some wierd chat. How can in retain the UTF chars in my script?

e.g. ||||||||

Any help will be appriciated.

Thanks,

2

Show your code please. Without that I'd only be making wild guesses.

3

You will need to make sure that your terminal/editor uses an utf8 locale. The locale command will show your settings and specifically you want to set LC_CTYPE to an utf8 locale (for example en_US.UTF-8).

How to go about it depends on your system (for debian linux: https://wiki.debian.org/Locale).

4

$ locale

LANG=
LC_CTYPE=en_US.UTF-8
LC_NUMERIC="C"
LC_TIME="C"
LC_COLLATE="C"
LC_MONETARY="C"
LC_MESSAGES="C"
LC_ALL=

When I try to post UTF chat.. in a vi editior. It still changes to

.
5

If you save that and view it with xdd / od -x , does it come up as 2e ? If it saves as two bytes instead, it is a font problem.

6

hi varun,

  • what is your unix system ?
  • are you in console or term utility ?
  • and please check endoding def for vi ? (:set fileencoding)
7

Hi, I am using a solaris machine.

I am using a terminal (Putty)

set fileencoding is not a valid options.

Please help I really need to get this going

8

Hi,
what is your "Controlling character set translation" settting on the Putty for received data from server (you)?

(putty->window->translation )

is it UTF-8 or ? (must UTF-8)

regards,
ygemici

9

Please show us the (sensibly abbreviated) output of od -ctx1 on both the original and the copied file.

10

Ok, I changed my setting in putty->window->translation and it worked. As soon as i did sudo to application user it again stops working so something is getting change.. not sure how to get it working over there as well.

11

Also when i cat the file i see the UTF chars but when I do a vi..I just see numbers not the actual UTF CHars.

Any help will be appreciated.

---------- Post updated at 10:28 AM ---------- Previous update was at 10:28 AM ----------

12

If you don't receive an answer immediately, WAIT! We are not "on call".

What is the LOCALE variable set to in your home environment, and in the application user?

It's possible the version of vi you have doesn't support UTF-8.

13

you must save the putty session.

---------- Post updated at 10:25 PM ---------- Previous update was at 09:27 PM ----------

Corona688 has been said already , probably your vi does not support it.
tyr different editors. (for ex -> vedit)

if you want force to vi working then maybe ,

1- ) Putty Translation Settings (Utf-8)
2 -) Solaris has UTF-8 for locale
what did you charmaps ?
locale -m

now you can see the UTF-8.. if is not in there then you execute

/usr/bin/uncompress /usr/lib/localedef/src/en_US.UTF-8/charmap.src.Z
/usr/bin/uncompress /usr/lib/localedef/src/en_US.UTF-8/localedef.src.Z

than try-again

locale -m
...
iso_8859_1/charmap.src
en_US.UTF-8/charmap.src
...

3 -) export LC_CTYPE="en_US.UTF-8"
(if `locale -m` output doesnt get the UTF-8 , you dont the set this locale already )

4- ) then try in the vi

regards,
ygemici

14

I have below SQL that I wasnt to run from a solaris machine using sqlplus

select * from TABLE_NAME where regexp_like(field_value, '|||||||||||||||||||||||||||||||||||||||||||||||' );

When I cat the file UTF char shows perfect however when i run the script from sqlplus it returns nothing. If you run it from sqldevloper it actually has these UTF char which should return rows.

Can someone help on what do I need to in order to get it working from sqlplus?

15

what is your Lang settings for sql ?

SQL> HOST ECHO $NLS_LANG