Perl script backspace not working for Unicode characters

Hello,

My Perl script reads input from stdin and prints it out to stdout. After I read input I use BACKSPACE to erase characters. However BACKSPACE does not work with Unicode characters that are multi-bytes. On screen the character is erased but underneath only one byte is deleted instead of all bytes of the character, which makes remaining bytes incomplete UTF-8 characters. Here's my script:

#!/usr/bin/perl
binmode STDIN, ":encoding(utf8)";
binmode STDOUT, ":encoding(utf8)";
my $str;
while(1) {
  printf("please enter text > ");
  chomp($str = <STDIN>);
  printf("You entered: %s\n", $str);
}

I am using bash in a Putty terminal with UTF-8 encoding. Everything on Unicode works fine in shell including BACKSPACE. It is only a problem in my Perl script. Can anyone help? Thanks.

-Tom

How old is the PERL? It seems to be ignoring the UTF-8 encoding mode. Some googled up articles say backspace should be mapped like delete ^? not ^H! Another suggests issuing the "stty iutf8" command. I thought the tty cooked handling was in the system, not PERL -- PERL just gets a line! You might run it in truss/strace/tusc to see what is passed.

http://sourceforge.net/tracker/index.php?func=detail&aid=2995319&group_id=67789&atid=518973
http://www.mail-archive.com/linux-utf8@nl.linux.org/msg01645.html

1 Like

Thank you very much for the pointers. I followed tips from 2nd link and added the line in my script:

system('stty iutf8');

It worked but not completely. Normal width characters worked fine, like ����. But wide characters like Chinese, one BACKSPACE erased all bytes correctly (3 bytes per char), but cursor moved back only one term cell instead of two cells for wide chars. The result was that some Chinese characters were still left visible on terminal even after all of the underlying bytes were deleted.

Well, it is in the realm of stty cooked output and terminal type accuracy. The maintenance of screen display is outside the application, even PERL, too. It is the flip side. I wonder if there is an 'outf8' for stty?

dtterm, xterm and Terminals Capable of Input and Output of UTF-8 Characters (International Language Environments Guide)

The Unicode HOWTO: Locale setup

UTF-8 and Unicode FAQ