Hello everyone,I'm reading a book and there's code fragment:
exec 3>&1
ls -l 2>&1 >&3 3>&- | grep bad 3>&-
exec 3>&-
It says that the red part of that code does not close fd 3 but the green does close the fd 3.I can't understand that.....Why?Any predicate will be appreciated.
Maybe the right side runs first, locally and the left side is a sub-process?
It looks to me that both red and green are closing fd 3. What book are you reading?
It'd be more accurate to say that grep and ls get separate sets of file descriptors, so they close them individually. The shell forks twice, once to create each of those processes. FD 3 is closed in the forks but not in the original shell itself. Once they return, the exec closes FD 3 in the shell itself. That's what the exec does, changes the original shell's own file descriptors.
I do agree with you,the book named "Advanced Bash-scripting Guide"
Thanks for your reply.What your said is the same as what i thought.We all think that FD 3 is closed in each process while the book said the process which executed the "ls" did NOT closed that.Confused...:wall:
http://www.tldp.org/LDP/abs/html/io-redirection.html
In my opinion, that's simply a poorly-worded comment on their part. fd 3 is closed for both. Perhaps what the author was trying to convey is that ls still has access to fd 3's destination because it was dup'd before closing (the >&3 before 3>&-). But, then, so does grep, since its inherited stdout leads to the same place as fd 3.
My advice would be to not get hung up on this example.
Regards,
Alister
Thank you so much.
Actually it now mostly makes sense. Notice the comment after the line in question.
ls -l 2>&1 >&3 3>&- | grep bad 3>&- # Close fd 3 for 'grep' (but not 'ls').
# ^^^^ ^^^^
# 1111 2222 3333
The item which I numbered 1111 closes fd 3, but the ls command had already made use of fd 3. So the process, which will become the ls command, uses fd 3, closes fd 3, and then forks. The child process, which will become grep, springs into existence with no fd 3. Why the author put in sequence number 3333 is not totally clear. But we are not really guaranteed that the grep process will be a child process of the ls process. If the shell was to be rewritten so that the grep process begat the ls process we would be closing fd 3 late. So while sequence 3 is unneccessary today it might be a hedge against a future revision to the shell or to make it more portable. Another possible reason for the superfluous sequence 3333 is to make it totally clear to the reader that grep process has no access to fd 3.
But in any event, the comment was meant to pertain to sequences 1111 and 2222. That's why the line is there.
With all due respect to your seniority and all the great, informative posts you have contributed long before I was even a gleam in the forum's eye, I believe your analysis of this pipeline and its redirections to be off the mark.
I will be using "the main shell" to refer to the shell tasked with interpreting the ls-grep pipeline.
The grep process will never be a child of the ls process. And, the ls process will never be a child of the grep process. They will be siblings, each a child of the main shell. Further, the order in which they are created does not matter.
For grep to beget ls would require more than a shell rewrite, it would require a grep one too.
Actually, the fork will happen before the i/o redirection, then an exec. If the i/o redirection were done before forking, file descriptor handling would get very very messy, as the parent would have to juggle things around before the fork, and then restore sanity afterwards (assuming the parent has further business to tend to).
As I understand it, the chain of events to realize the ls-grep pipeline is as follows:
The main shell will first create the descriptors that the pipe will use. Then, it will fork twice (creating two subshells). One of the subshells will eventually exec to become ls and the other will eventually exec to become grep.
One subshell will modify stdout to point to the pipe (this subshell will become ls) and the other will modify stdin (for grep). Before exec, these subshells have to process each command's command-line i/o redirections. In the case of the subshell that will become ls, this would make fd 1 a copy of fd 3 and would then close fd 3. In the case of the subshell to become grep, there's nothing to be done but close fd 3. Now, when each subshell exec's, there is no fd 3 available to either process.
Agreed. The same is true of ls.
If I'm the one that's off the mark, I await the Cluestick of Enlightenment 
Regards,
Alister
Not really true. I actually said that grep was the child and ls the parent. You made a slight goof and reversed my comments. But an experiment shows that the reversed statement you attributed to me is true. So I hereby adopt it as what I meant to say.
$ /bin/sleep 60 | grep bad &
[1] 32429
$ ps -f
UID PID PPID C STIME TTY TIME CMD
xxxxx 3417 3416 0 Mar08 pts/1 00:00:00 -ksh
xxxxx 11703 3417 0 Mar08 pts/1 00:00:00 ksh
xxxxx 32424 11703 0 13:42 pts/1 00:00:00 ksh
xxxxx 32429 32424 0 13:43 pts/1 00:00:00 grep bad
xxxxx 32430 32429 0 13:43 pts/1 00:00:00 /bin/sleep 60
xxxxx 32431 32424 0 13:43 pts/1 00:00:00 ps -f
$
The grep process wound up as parent and the sleep process wound up as child. I would not want to bet that it always works that way. And no I don't have a special grep process that knew to establish the child.
32424 forked to create 32429 but both processes are still ksh because neither has exec'ed yet. 32429 then forked again to create a child 32430. At this point the whole pipeline exists and both processes exec'ed their respective programs.
If I recall correctly, only the AT&T ksh and zsh run the last command in a pipeline in the current shell environment.
Edit: Irrelevant, see further comments.
Only if that command is a shell built-in. I did not know that zsh does that too. Now that bash supports co-processes this issue is the primary reason I don't want to switch to bash.
It's interest that bash is making the main shell the parent of each process in the pipeline. I guess it needs this to support it's PIPESTATUS concept. This must make hard to get the entire pipeline into a separate process group.
BTW, regarding the original question, my admittedly flawed, reasoning still might be what the author of that example was thinking. If not, then I not know what he was thinking. Anyone got a better idea on that?
Yes, of course, thanks for pointing it out.
I stand corrected on my statement that grep and ls will be siblings. They usually are, but its an unprescribed implementation detail. I think my analysis is generally sound, though.
For kicks, I tested your sleep-grep pipeline on a few shells (old versions on a disused laptop); for each shell, I ran the pipeline in the background and in the foreground. att ksh93 backgrounded was the only one of six scenarios which showed that parent-child child relationship.
att ksh93: bg: parent-child
att ksh93: fg: siblings
bash: bg: siblings
bash: fg: siblings
pdksh: bg: siblings
pdksh: fg: siblings
Regards,
Alister
Interesting data and I'm glad you caught my error. But this still leaves the original question up in the air. What are your thoughts on the example the OP asked about? There doesn't seem to be much hope of saving it. 
I don't know, I asked on c.u.s., this is the releveant part of the Advanced Bash-Scripting Guide:
# Redirecting only stderr to a pipe.
exec 3>&1 # Save current "value" of stdout.
ls -l 2>&1 >&3 3>&- | grep bad 3>&- # Close fd 3 for 'grep' (but not 'ls'). #
^^^^ ^^^^
exec 3>&- # Now close it for the remainder of the script.
# Thanks, S.C.
I think S.C. stands for Stephane Chazelas, he's one of the most respected contributors there on c.u.s. (comp.unix.shell).
---------- Post updated at 11:05 AM ---------- Previous update was at 07:41 AM ----------
Got an answer from Icarus Sparry.
Wow. Duh! That seems so obvious now. Personally, I wasn't able to make sense out of " # Close fd 3 for 'grep' (but not 'ls'). # " because I took it as referring to the entire pipeline, not just the final component (just like, I suspect, the OP).
It wasn't going to keep me up at night, but thank you, radoulov, for showing us the way out of the box 
Regards,
Alister
Me too
You're welcome 
Thanks all of you,you guys are so kind!!And i love this forum more because of you guys!