Hi,
While working with exec in find , I have this query.
Exec when used in while loop , (suppose reading lines from a file and using exec to echo something more to the line) , will exit on first line after successful execution as it exits the current shell.
But the same thing do not happen with find command. Find is also working on each file while finding them and exec is executed for each file (in case of *), so why don't exec causes find to terminate on the first find as it does in while loop.
Does find creates a sub shell for each file which means if i have 100 files (I gave * in find) then 100 sub shells will be created one after other (previously being replaced by exec) with each executing exec (i.e 100 new sub shells also created by exec one after other )
Thanks,
Yes, with -exec the find (for each matching filename) forks another find instance (not shell instance) that execs to the command. The primary find process continues.
1 Like
@MadeInGermany would you please clarify / confirm that the parent find instance forks a child instance of find which then calls the -exec.
I'm trying to understand why there is the interstitial find instance and why the parent find doesn't call the exec directly.
I happen to have a find running across an old Maildir compressing files with -exec;
find /path/to/Maildir -name '*.hostname.*' -exec gzip '{}' \;
And ps output tree is my ${SHELL}, find ..., and gzip.
I don't see the interstitial find process that you're talking about.
That being said, I suppose there is a chance that parent find instance is forking itself and then the child find instance is replacing itself with the value of exec.
Thank you for helping me understand and learn.
Each time it does the fork, where the child instance immediately execs to the command.
You must have great luck to ever see two find processes...
BTW the shell does the same fork/exec with every external command. An explicit exec just skips the fork. Some shells are optimized, they skip a fork with the last command in the script.
1 Like
Thank you for clarifying @MadeInGermany. 
#TIL
further to teammate @MadeInGermany , check out the following trace files from two distinct invocations of a simple find command
# this variant of find will daisy chain results an pass them to whatever -exec command is
# typically resulting in a single forked process.
#
strace -e trace=process,file -ff find ../3932?[26] -name title -exec stat -t '{}' + 2>/dev/null
../393216/title 359 8 81b4 1000 1000 805 5262370 1 0 0 1709737627 1707466469 1707466469 0 4096
../393222/title 486 8 81b4 1000 1000 805 5251765 1 0 0 1709737627 1707469882 1707469882 0 4096
# see the 'single' file for details
#
# this variant '\;' will spawn/fork a process for each result
#
strace -e trace=process,file -ff find ../3932?[26] -name title -exec stat -t {} \;
../393216/title 359 8 81b4 1000 1000 805 5262370 1 0 0 1709737627 1707466469 1707466469 0 4096
../393222/title 486 8 81b4 1000 1000 805 5251765 1 0 0 1709737627 1707469882 1707469882 0 4096
# see the 'multi' file for details
snippet from the find man page about the behaviour of 'command '{}' +' and command '{}' ;
find.info (1.8 KB)
multi (6.8 KB)
single (4.6 KB)
Thanks all for joining discussion and valuable sharing.
But why can't an instance of While loop (say, with a cat command ) work like find, both as external commands initially executes a sub-shell and then the functionality is different , so what causes find to fork new process if exec is there and Why while loop with a command (say cat) will not create a separate process on encountering exec.
@dextergenious , show coded examples of what you are after - verbalising leaves too many chances for misinterpretations/confusions , whereas actual code should leave no ambiguties, that can be supplemented by questions wrt behaviours.
tks
In the shell a simple while loop runs in the main shell, so an exit or an execed command that terminates will terminate the main shell.
Connecting the while loop's input or output to a pipe enforces a sub shell, and the behavior changes. For example
cat myfile | while read line
do
...
done
You can always put shell code in a sub shell, using parentheses:
(
# sub shell starts here
while ...
do
...
done < myfile
echo "last sub shell action"
)
echo "back in the main shell"
and within the sub shell an exit or an execed command that terminates will terminate the sub shell - the main shell continues.
BTW
exec echo "terminating"
runs (exec without fork) the external /bin/echo command.
More efficient is a command group
{ echo "terminating"; exit; }
The echo and exit are builtin commands, don't need a fork/exec.
The shell tries to run a command group in the main shell, but a pipe from or to the group would enforce a sub shell.
Both command group and while loop are code blocks. Also a for loop, a case-esac, a if-fi are code blocks.
Example redirected if-fi:
if [ "$a" = "head" ]
then
head
else
tail
fi < /etc/group
1 Like
@munkeHoller
Thanks for your reply,
With that understood , I used the following code in which I have used another while loop apart from the {} to create 2 nested sub-shells within main While loop which runs in main shell.
So with this code, exec should terminate and return to the main while loop which again calls the exec for 2nd line and this should continue up to EOF.
But still exec is causing the main shell to terminate and I only get first line
<
while read line
do
{
{
while(true)
do
exec echo hello $line
done
}
continue
}
done < file
File contents :
line 2
line 3
line 4
O/p as to be :
hello line 2
hello line 3
hello line 4
O/p as is :
hello line 2
Apology ... My last post was @MadeInGermany ..
Enforce a sub shell using parentheses ( ) - not { } that avoids a sub shell.
while read line
do
{
(
while(true)
do
exec echo hello $line
done
)
continue
}
done < file
And you don't need the { } here, neither the continue
Yes, I was using wrong braces. Now it simply reduced to :
while read line
do
(
exec echo hello $line
)
done < file
Thanks all for your valuable inputs in discussion.
From the knowledge sharing in this thread;
I just share with you my script which takes string as an argument and searches it in all the files whose names are listed in a file - 'fileofFLENAMES' .
This is similar to using exec with find to grep text from file but here we are using while loop with exec (so I suppose it has less process overload than find) .
It lists the names of files in which the string is found and shows message " found in " after the each match of file
Code:
#!/bin/bash
while read line
do
(
exec cat $line | grep -w "$1" && echo "(found in $line)"
)
done < fileofFILENAMES
Files data:
File1:
you are good
hello dear
File2:
you are bad
no hello dear
File3:
you are neither good nor bad
File4:
fileofFILENAMES:
file1
file2
file3
file4
Execution:
./myscript "no hello"
no hello dear
(found in file2)
./myscript "hello"
hello dear
(found in file1)
no hello dear
(found in file2)
That is an explicit fork to a sub shell then an explicit exec (that's an exec without a prior fork).
But fork then exec is the default for all command invocations in the shell; simply do
while read line
do
cat "$line" | grep -w "$1" && echo "(found in $line)"
done < fileofFILENAMES
A further optimization is to eliminate the (fork/exec of) cat
while read line
do
grep -w "$1" < "$line" && echo "(found in $line)"
done < fileofFILENAMES
Explanation:
The cat reads the file, passing it to stdout that the pipe connects to the stdin of grep
The < lets the shell open the file as the stdin of grep, so grep directly reads from the file.
1 Like
Does the following also creates another sub shell each time the loop executes
<
cat "$line" | grep -w "$1" && echo "(found in $line)"
and in
cat "$line" | grep -w "$1" && echo "(found in $line)"
also ,will sub shell is created each time
Thanks ,
Each invocation of an external program does a fork/exec: cat, grep. Not echo because it is a builtin command. The forked shell execs immediately, one cannot name it a sub shell.
And the ( ) are literal characters because they are inside a " " string.
So, will both the below codes will produce 100 forks and 100 exec calls for 100 files to work upon
find . -name * -type f -exec grep "sometext" {} \;
and
while read line
do
(
exec cat $line | grep -w "$1" && echo "(found in $line)"
)
done < fileofFILENAMES
Thanks for all the help
Quote the * so the shell does not glob-match it, but passes it to find dequoted. Then find does the glob matches.
Yes, 100 fork/exec for 100 matches.
A + does only one fork/exec:
find . -name "*" -type f -exec grep -h "sometext" {} +
The one grep is called with 100 collected arguments (file names).