ksh bug?

I think I discovered a bug in ksh.

$ cat test.sh
x=`echo hello | cat 2>&1 >/dev/null`
echo $x
y=`echo hello | cat >/dev/null 2>&1`
echo $y

works with all shells, including ksh in normal mode.
But does not work with ksh -x :

$ ksh -x test.sh
+ + echo hello
+ cat
+ 2>& 1 x=1> /dev/null
+ echo 1> /dev/null
1> /dev/null
+ + echo hello
+ cat
+ 1> /dev/null 2>& 1
y=
+ echo

sh -x or bash -x or zsh -x do it correctly:

$ bash -x test.sh
++ echo hello
++ cat
+ x=
+ echo

++ echo hello
++ cat
+ y=
+ echo

I guess I don't see what's wrong.

It looks like ksh is running the parts of the pipelines in different threads (so you see jumbled output in the trace), but both x and y are assigned empty values and the echo $x and echo $y seem to have done the right thing. What am I missing?

If you mean that the trace output from bash and zsh don't show redirection operations performed by the shell, I don't think that is a ksh bug.

x definitely gets the value "1> /dev/null" assigned when run with "ksh -x".
The lines without a + are actually printed on the screen.

I had worked on a script where a "ksh -x" run produced a total different result, compared to a normal run with "ksh".
Here is another variant of the above script:

$ cat test.sh
x=`echo hello | cat 2>&1 >/dev/null`
if [ -n "$x" ]; then
  echo WRONG
else
  echo RIGHT
fi
y=`echo hello | cat >/dev/null 2>&1`
if [ -n "$y" ]; then
  echo WRONG
else
  echo RIGHT
fi
$ ksh test.sh
RIGHT
RIGHT
$ ksh -x test.sh
+ + echo hello
+ cat
+ 2>& 1 x=1> /dev/null
+ [ -n 1> /dev/null ]
+ echo WRONG
WRONG
+ + echo hello
+ cat
+ 1> /dev/null 2>& 1
y=
+ [ -n  ]
+ echo RIGHT
RIGHT

OK. With this example, I'm seeking the same problem when using ksh version:

  version         sh (AT&T Research) 1993-12-28 s+

on OS/X. With this version the trace looks like:

+ cat
+ echo hello
+ 2>& 1 + x='1> /dev/null'
+ [ -n '1> /dev/null' ]
+ echo WRONG
WRONG
+ cat
+ echo hello
+ 1> /dev/null 2>& 1
+ y=''
+ [ -n '' ]
+ echo RIGHT
RIGHT

I can confirm the results from post #3 using 93u+ 2012-08-01 .

Regards,
Alister

---------- Post updated at 05:43 PM ---------- Previous update was at 04:48 PM ----------

Taking a closer look at this, and experimenting with various oneliners, I don't think this is a bug.

ksh writes the trace to stderr and stderr is redirected by the scripts above.

$ bash -xc 'echo foo 2>&1 >/dev/null bar' > stdout
+ echo foo bar
$ cat stdout
$
$ ksh -xc 'echo foo 2>&1 >/dev/null bar' > stdout
+ echo foo bar
+ 2>& 1 $ cat stdout
1> /dev/null

bash doesn't trace redirections. ksh does. Redirections are set aside during parsing and are processed after the other command steps. Just before processing the 2>&1 redirection, ksh prints it to stderr. Then stderr is redirected. Further redirections appear on stdout. In my example, that's the file named stdout ; in MadeInGermany's script, it is the pipe leading from the command substitution to the parent shell.

Regards,
Alister

5 Likes

Then let's call it a shortcoming.
It looks like -x mode uses stderr, and can conflict with redirection in the shell code.
Then, why stderr and not another file descriptor?
Maybe that was not available on all Unix system 1988?

Probably because that's exactly what stderr is there for. Being 'clever' about it would make obvious things, like redirecting a trace's output into a file, difficult. A shell can't make too many assumptions about what you intend to do with it.

You, the programmer however, are allowed to make more assumptions. Why were you using the same file descriptor as the debugger?

1 Like

It can get messy, yes.

This is also an issue for bash ...

$ bash -xc 'echo foo; exec 2>&1; echo bar' > stdout
+ echo foo
+ exec
$ cat stdout
foo
+ echo bar
bar

... but since it does not trace redirections, the destination of the trace won't change mid-command.

I would assume because stderr is known to be reserved for diagnostic and error messages, not data. If an alternative file descriptor were chosen, the possibility for a collision would still exist.

Corona688 makes an excellent point:

Cleverness would include attempting to detect and handle a collision between the internal tracing descriptor and a script's redirections.

Regards,
Alister

Thank you Alister and Don!
I have been working on a script that captures failures from a "df".
Everything fine, until I wanted to quickly find a mistake in the post processing.
The difficulty was reason enough for me to change to #!/bin/bash

As much as I like bash over ksh, I'm not sure that's a good reason. It uses stderr for trace output too.

stderr=`df 2>&1 >/dev/null`
case $stderr in
*"Permission denied"*) ...
*"not a block device"*) ...;;
...
esac

Any suggestions how to recode this?

How about nohup?