Named pipe hanging?

Ok, I can't seem to figure this out or find anything on the web about this.

I'm on Sun Solaris, UNIX.

I have the following test script:

#!/bin/ksh

touch test.file

LOG=./tmp.log
rm -f ${LOG}

PIPE=./tmp.pipe
mkfifo ${PIPE}
trap "rm -f ${PIPE}" EXIT

tee -a ${LOG} < ${PIPE} &

echo "Hello world"  >${PIPE}  2>&1

ls -ltr test.file

echo "Goodbye world"  >${PIPE}  2>&1

which hangs on the final echo command when I run it.
Interestingly, if I comment out the ls, or either echo command, it works fine.

So I'm trying to understand what's happening here. Hoping somebody could explain?

What I'm trying to do:

• capture stdout and stderr into a log file. [edit] correction: want to capture stdout and sterr to both display and log file[/edit]
• need to capture the return code of the command as well (ie $?) (hence why I'm mucking around with PIPE's so I can displace the tee command).

It seems to work sometimes, and then hang on other occasions, and I haven't found a pattern to it yet.
Hoping anyone can shed light on this - or provide a suitable work around.

Thanks!

What return code are you trying to capture?
What are you trying to capture in the log file? (What you have seems to want to capture the output from the echo commands, but not the output from the ls command.)
It would seem that:

#!/bin/ksh

touch test.file

LOG=./tmp.log

echo "Hello world"  >${LOG}  2>&1

ls -ltr test.file

echo "Goodbye world"  >>${LOG}  2>&1

would be much simpler than what you're doing and produce the same results.

Are you trying to create a dual pipeline...if so you cant displace the tee command...

I want to display output and stderr to both display and the log file. [edit] sorry, original post forgot to mention display [/edit]
I need to use tee for that.
I also want to capture the return code of each command (ie echo, ls, cp, whatever). If I use:
<command> 2>&1 |tee -a <log>
then when I check $? I get the return code for tee.

So I found the logic for named pipes, and the above example seemed to work most of the time.
Then I hit this "side case?" Not really sure.

FYI: I did just find something on the web that helped.

Changed my code to this (added the exec), and now it works
(soo confused. )

#!/bin/ksh

touch test.file

LOG=./tmp.log
rm -f ${LOG}

PIPE=./tmp.pipe
mkfifo ${PIPE}
trap "rm -f ${PIPE}" EXIT

tee -a ${LOG} < ${PIPE} &

trap "exec 5>&-" EXIT
exec  5>${PIPE}

echo "Hello world"  >${PIPE}  2>&1

ls -ltr test.file

echo "Goodbye world"  >${PIPE}  2>&1

From what I understand, the named pipe thinks that after the ls (or cp, or mv), that I'm all done with it. (this occurs whether I send the output of the ls, cp, or mv to the PIPE or not). By throwing the exec 5>$PIPE in there, it seems to trick it into expecting more input ???

The shell tries to open the pipe before it puts the command in background, and freezes waiting for it. You have to put it in a separate subshell to avoid this.

( command < pipe ) &

It still hangs, until the other end opens, but it's some other shell hanging, not yours.

What is your goal here, though?

Yes, the tee command, I run using & to throw into background.
As I indicated, if you comment out the ls (which seems to have nothing to do with the PIPE - it works fine - no hang.
With the script as is - the first echo makes it into the pipe, the ls works fine - displays to screen only.
then the last echo just hangs. Yes - my script is hanging.

What I'm trying to do:

• want to capture stdout and sterr to both display and log file
• need to capture the return code of the command as well (ie $?) (hence why I'm mucking around with PIPE's so I can displace the tee command).

So, try:

#!/bin/ksh
touch test.file
LOG=./tmp.log
(
        echo "Hello world"
        ls -ltr test.file
        echo "Goodbye world"
) 2>&1 | tee ${LOG}

You'd use a named pipe only to send the output of the preceding pipeline to another command as the below lines of shell code show...

mkfifo somepipe   # create a named pipe
wc -l < somepipe &   # start a reader process
ls -ltr | tee somepipe | grep "some_file_name"   # pipe output of "ls -ltr" to "grep" via tee and to "wc -l" via somepipe

but the stuff you're trying to do doesn't need a fifo and I'd go with what Don Cragun suggests...

Is it okay if the streams are merged before they're printed, or does stderr still need to be redirected outside the script?

wc -l < somepipe & will hang in a shell, while wc -l somepipe & will not... The difference is subtle but important: One tells wc -l to open the pipe, which happens after it's run and backgrounded. The other tells the shell to open the pipe, which happens before it's run and backgrounded.

Nope...the only difference is that wc -l somepipe & will give the no. of line plus the filename whereas wc -l < somepipe & will give the no. of lines in the ls -ltr listing...that is just the way wc -l works...

Interesting... I'm guessing the strange hangs I've had trying to do that with stdin/stdout are effects of backgrounding something talking to the terminal, then.

I tested your and my versions and that is the only difference I saw...

Probably. & implies a subshell.

Regards,
Alister

Ok, so now at the end, check/show the return code of the ls command ?

#!/bin/ksh
touch test.file
LOG=./tmp.log
(
        echo "Hello world"
        ls -ltr test.file
        echo "Goodbye world"
) 2>&1 | tee ${LOG}

echo "Return code is: ???"

I've tried just capturing it inside that block, but accessing it outside the brackets fails (null value).

You said you wanted to capture everything in the script in the log file. So you've changed the rules again... grumble, grumble, grumble...

#!/bin/ksh
touch test.file
AFTERLOG=./$$.after.log
LOG=./tmp.log
(
        echo "Hello world"
        echo "1st echo return code is: $?" > "$AFTERLOG"
        ls -ltr test.file
        echo "Return code is: $?" >> "$AFTERLOG"
        echo "Goodbye world"
        echo "2nd echo return code is: $?" >> "$AFTERLOG"
) 2>&1 | tee "${LOG}"
cat "$AFTERLOG"
rm -f "$AFTERLOG"

This is what has been happening in your original code ...

When tee opens the fifo for reading, if no other process has yet written to the fifo, tee will block (hang/sleep) until another process opens the fifo and writes to it.

Each echo statement will attempt to open the fifo, write to it, and then close it. Most likely, when this succeeds, it is because both echoes have written to the fifo before tee has a chance to read it.

Adding ls greatly increases the chances of tee reading between the echoes. If that happens, after reading the first echo's text, tee will detect EOF and exit. When the second echo opens the fifo for writing, it will hang until something opens the fifo for reading (which will never happen with your sample script).

exec 5>fifo prevents the hang by ensuring that there is always a write file descriptor for the fifo. That descriptor is never used, but since it has permission to write, so long as it exists, tee will not read EOF; instead, tee will block until someone writes (in this case, the second echo).

Regards,
Alister

apologies, I only changed the rules once I editted my original post after your first post, and clarified the two things I'm trying to do.

hmm, that "sort of" works, but then I have to go digging the error code out of the script and stuff it into a variable if I need to use it / check it.

Thanks ... I think what I found with the exec 5>&PIPE seems to work more consistently.
And it seems from the responses, that another (cleaner) option probably doesn't exist.

Don't want to take any more of your time Sorry for confusion, thanks for help. I think I have something working, and I think I understand what's going on.

Cheers!