Why stderr file descriptor redirection makes ksh's "select" construct hang.

kchinnam · May 6, 2010, 10:50am

I am trying to use one global declaration --> "exec 2>$ERR" to capture all stderr outputs that may occur anywhere in my script.
Then close it at the end of the script using --> "exec 2<&-"

I am using KSH on Solaris 8.
KSH Version M-11/16/88i

If I comment two "exec .." statements in the script,, script[select] works fine.. but if I use --> "exec 2>$ERR" script[select] is hanging.

test.sh

#!/usr/bin/ksh

ERR=all_script_errors.log

# 1. My intent here is to use one global declaration to capture stderr outputs that may occur anywhere in my script.
# 2. For the scope of this script(PID), stderr will be routed to ERR file
exec 2>$ERR

PS3='Select an option and press Enter: '
select i in Date Host Users Quit
do
   case $i in
   Date)  date;;
   Host)  hostname;;
   Users)  who;;
   Quit)  break;;
   esac
done

# close/Reset stderr global error redirection.
exec 2<&-

## EOF

Please try the script on your end..

vidyadhar85 · May 6, 2010, 11:04am

try running the script with sh -x scriptname you will get an idea

methyl · May 6, 2010, 11:23am

Extract from "man ksh".

    select identifier [in word...] do list done
                     A select command prints on standard error (file
                     descriptor 2),

You are intercepting standard error.
Just for interest if you type "4" while it is apparently hung, the script exits tidily and the menu text appears in the log file.

kchinnam · May 6, 2010, 11:42am

Hi Vidyadhar here is the output..

$> sh -x test.sh
ERR=errors.log
+ exec

I am asking experts here to try the code because its tricky..
Here is the same code without "select" construct..

test1.sh

#!/usr/bin/ksh
set -vx

ERR=all_script_errors.log

# For the scope of this script(PID), stderr will be routed to ERR file 
exec 2>$ERR

blabla

# Reset error redirection. Since select construct's output is going to STD_ERROR.
exec 2<&-

## EOF

Please try the code on your end..

---------- Post updated at 11:42 AM ---------- Previous update was at 11:27 AM ----------

Methyl you are right,, Select statement is printing menu choices to stderr.

After I closed stderr redirection,, just before select construct gets executed, now select menu is neither printing output to console nor to $ERR ..!!
Does that mean my stderr descriptor closing(exec 2<&-) is wrong?
#!/usr/bin/ksh

ERR=all_script_errors.log

# For the scope of this script(PID), stderr will be routed to ERR file 
exec 2>$ERR


# Reset error redirection. Since select construct's output is going to STD_ERROR.
exec 2<&-

PS3='Select an option and press Enter: '
select i in Date Host Users Quit
do
   case $i in
   Date)  date;;
   Host)  hostname;;
   Users)  who;;
   Quit)  break;;
   esac
done

## EOF
<QUOTE>
$>more all_script_errors.log
<None>

With --> "exec 2<&-" statement at the end of the script,, I get following output confirming your observation..

[CODE]

kchinnam · May 7, 2010, 1:18pm

Please could someone check -->
why even after closing stderr File Descriptor using (exec 2<&-), select construct could not print to console ?

alister · May 7, 2010, 1:31pm

That does not reset stderr to its original location. That closes the file descriptor and it can no longer be used.

The most correct approach is to "save" the original destination of stderr in case you need it later. For example:

exec 3>&2                  # File descriptor 3 now points to the original destination of stderr
exec 2 > someotherplace    # Standard error is now redirected for all commands that follow
select ...
do
...
done 2>&3    # Select's stderr is redirected to the original stderr destination, while
             # all other commands continue to redirect stderr to "someotherplace".

Regards,
Alister

kchinnam · May 7, 2010, 1:48pm

Alister, your suggestion is working..

How can I reset only this--> exec 2>$ERR, and not worry about breaking script that follows by closing stderr --> exec 2<&- ?

Is there a command that I can use to check if stderr is closed ?

alister · May 7, 2010, 2:09pm

It seems I was busy editing my response after you had replied. I'm sorry. I hadn't noticed. I tried to restore it so your mention to my suggestion makes sense.

If the original location of stderr is stored in file descriptor 3, you can permanently restore it using:

exec 2>&3

With regard to testing the validity of a file descriptor (in this case, stderr), I suppose you can attempt to duplicate a descriptor and then test the exit status of the exec:

if exec 5>&2; then
    exec 5>&-     # If the exec succeeded, fd 2 is alive and well.  Close fd 5 since we don't need it.
else
    echo stderr is closed
fi

In that script, 5 is assumed to be an unused descriptor. If it is not, choose another number. Also, that should only work with interactive shells. In non-interactive shells, an exec failure will cause the shell to exit (assuming it's compliant with the posix standard sh).

If anyone knows of a better, more elegant, way to do this in sh, please do share your expertise (if it's not a portable solution, please mention in which environments you know it to work).

Regards,
Alister

kchinnam · May 7, 2010, 4:14pm

Alister thanks for sharing your ideas,, they are working, it solves my current problem..

But I am still frustrated with this redirection concepts.. Why in the world select construct was implemented to use stderr to print to console !??

Whatever select construct creator's vision was,, aren't we defeating that purpose if we route "select ..; do .. done 2>&1" for the scope of select ?

How can we only stop stderr's file handle from pointing to $ERR temporarily and continue to use stderr as its intended for the scope of select?

how about something like-->

ERR=all_script_errors.log
exec 2>$ERR             # Route all errors from script to one file, easy to notify operations that something broke.
... <Tons of code, function calls, execute include files etc..>
..
exec 2>&-                # close fine handle since at least we know select wants to have control over STD_ERROR i.e 2
exec 2>&STD_ERROR  # ( in theory -:)  Let us say this can let stderr to go to console as it was intended by shell.
select ...
do  

  ... <Lot of code, god knows who wants to use STD_ERROR here..> ...

done                        # Now select will have no problem printing its error output to console.
exec 2>$ERR              # now go back to our good programming practices..

reason for my frustration is I have lot of scripts in production that use global error routing to error file..
I recently realized sometimes that is not working,, I wonder what other commands like select mess around STD_ERROR..!

alister · May 7, 2010, 5:01pm

All unix tools use stderr for error and informational messages, which includes prompting. Standard output is reserved for actual data. However, as you have seen, you are free to swap them around with redirections

As I said earlier, If you need to undo a file descriptor's redirection, you need to store the descriptor's original destination in another descriptor. I demonstrated how to do that in my first post in this thread (though it's possible you missed it as I kept editing what I included, again, sorry for that), Why stderr file descriptor redirection makes ksh's "select" construct hang. Post: 302419560.

Regards,
Alister

---------- Post updated at 05:01 PM ---------- Previous update was at 04:36 PM ----------

That 2>&1 suggestion was from an earlier version of my original post. You probably don't want to do that. The redirections applied to select apply to all commands executed within the select as well. Doing 2>&1 after the select's "done" will send the stderr of all commands to stdout, not just the select's prompt.

Either abandon the notion of globally redirecting stderr, or accept that you may have to append per-command redirections. In this case, if you don't want the standard error of the command executed by select to redirect to the same place as the select prompt (say, to a file instead of the screen), you will need to override the global stderr for the select statment. Then, within the select, you will need to restore the global redirect per command or command block. It's a mess.

A possible alternative, which logs all standard error (including select prompting), but allows the prompt to be seen on screen (along with any other error messages), can be relatively cleanly implemented from outside the script:

exec 3>&1
script.sh 2>&1 1>&3 | tee standard_err.log 1>&2
exec 3>&-

That works by using file descriptor 3 to store the original destination of stdout. Then when script.sh is executed, its stdout will be redirected into the pipe. However, we don't want that. We want its stderr to go into the pipe, so we redirect script.sh's stderr into the pipe, by copying stdout's current destination. Then, we point stdout to its old location which is stored in fd 3. The order of operations is very much significant. If "2>&1" and "1>&3" were reversed, nothing would flow into the pipe. Also, it's key to realize that redirections to/from a pipe occur before other redirections.

If that's confusing to you (which is understandable), the best advice I can give you is to read up a bit on redirections and tinker with them until the concept clicks. It's one of the more difficult concepts in sh scripting (particularly if the person isn't familiar with the kernel's notion of file descriptors and how they interact with system calls like open, close, dup, dup2, fork, and exec).

Regards,
Alister

kchinnam · May 7, 2010, 6:14pm

I am impressed with the depth of your explanation.

I am also totally disappointed that my vision of having one global error redirection is a bad idea.

You are right,, its hard for me to visualize how they will be working without trying them..

Is there a master reference from source that may explain how--> open, close, dup, dup2, fork, and exec work in ksh?

jlliagre · May 8, 2010, 4:59am

#!/usr/bin/ksh

ERR=inner_script_errors.log

PS3='Select an option and press Enter: '
select i in Date Host Users Quit
do
   case $i in
      Date)  date;;
      Host)  hostname;;
      Users)  who;;
      Quit)  break;;
   esac 2>>$ERR
done