sort file with non ascii chars and cjk with perl

Shell Programming and Scripting
1

Hello,
I am not a programmer, please be patient.
Actually, I have started to look into Perl because it seems to be able to solve all the problems (or most of them) I happen meet using my computer. These problems are generally all text-manipulation-related.

Although I started to study, I cannot figure out (yet?) how to sort a file with special characters in it.
With the Unix "sort" utility, I don't get the expected output.
For example, I need to sort a file where some words have a leading "�", which I need to be put just after the "normal s" and not after "z".

I had a look at the relevant (for what I understand) Perl tutorials, and it seems to be quite complicated for a new comer. Are there ready to use scripts?

For what concerns cjk, I've found at CPAN a tool I need to convert traditional to simplified Chinese and vice-versa (Encode::HanConvert), but cannot find a similar tool for sorting Chinese characters (by stroke, radical, pinyin). Does it exist or do I have to learn throughly Perl to do what I need?

Any suggestion will be appreciated.

2

Regular sort responds to a sort request buy comparing what is called a collation sequence. This is defined by locale settings.

What does this give for output? Please show it:

locale

Look at the variable named LC_COLLATE. That sets how sort sees this character comparison.

3

I don't know Chinese att all but a google search found this:

http://germain.its.maine.edu/~hiebeler/cedictscripts/cedictsort

4

sorry, forget to mention that I work on OpenBSD, so no locale support (but you can do all multilanguage work with unicode-aware apps and tools)

5

I previously probably searched with the wrong keys. I've found this at cpan:
Unicode::Collate
It seems to be what I need, I installed it, but sorry don't know how to use it inside a script.

6

this didn't help?

7

ok, actually I did read it but could not figure out how to use it (I'm really new to Perl and scripting in general). After trying many times, I managed to get my "�" right after the "s", but now they are both at the beginning of the list:

sss
���
aaa
aab
abc
bbc
lmn
mmn
zzz

this is how I did it:

use Unicode::Collate;
$Collator = Unicode::Collate->new(
       table => undef,
       entry => << 'ENTRIES',
0073  ; [.1137.0020.0002.0073]
0161  ; [.0000.0041.0002.030C]
ENTRIES
);
open (NAMES_FILE, "< path-to-my-file")  or  die "Failed to read file : $! ";
my @not_sorted = <NAMES_FILE>;
@sorted  = $Collator->sort(@not_sorted);
print @sorted;
close (NAMES_FILE);

Now, how do I tell Perl to leave the sorting order intact and to change only the part I need? Or should I probably say to insert my "�" just after "s"? Or do I have to make a complete sorting table to do it?