Book and Links about Shells; and zsh question

HI,
I would like to ask You about some good books or links where I can find information about shells, theoretical information.
I will be grateful if You can help me

And I have question about zsh loop

trivial script:

#!/bin/zsh
 for i in {1..100000}
 do
 echo $i;
 done

exec time is 10 sec
but for 500000 loops
exec time is 300 sec

Why for 500000 loops execution time is 300 seconds, it's only 5x 100000 loops it should be about 50 sec
In other shells it's OK

  1. there is not a lot of 'shell theory'. Production shops almost always choose a POSIX-compliant shell: probably one of: ksh (ksh88 ksh93 zsh) bash. The reason for this is portability.

read ISO/IEC subsection 3 ISO/IEC 9945
This tells you how shells are 'meant to be' I dunno if that is theory or not.

a. terminal i/o is slow, net traffic botlenecks may also be involved
try this on two large-ish files, use different files to circumvent file caching:

time cat bigfile1 
time cat bigfile2 > somefile

b. the primary cause of loops bogging is process creation. You are executing echo.
if it is /usr/bin/echo and not the shell builtin echo, every loop execs a whole new child process. This is a huge amount of overhead.

which echo

tells you whether the echo you ran is builtin or not.

We need more information on the platform and OS to give you anything more useful

When I type

which echo

it is /usr/bin/echo
Can I choose which echo would be executing ?

And why in different shells everything is OK ?

---------- Post updated at 01:09 PM ----------

Actually, I just now noticed the "zsh" part of the subject, checked and saw that zsh (which I have zero experience with) does have a "which" builti-in. My apologies, jim_mcnamara, for jumping the gun. I had assumed you were referring to the common (though I don't think posix standardized) which(1) executable found on most BSD and Linux systems.

I will leave my original post (which follows) as is, as I think it may prevent misunderstandings among those who (like me) do not use zsh.

---------- Posted at 12:35 PM ----------

That's incorrect. which(1) will search $PATH directories for an executable. If it finds anything, it will always be an external executable.

Use type to see what is actually being run.

$ which echo
/bin/echo
$ type echo
echo is a shell builtin
$ type -a echo
echo is a shell builtin
echo is a tracked alias for /bin/echo

The first result from the type built-in is what the shell will actually use. There are ways to use something other than the first found option, read your shell's man page.

Also, on ksh variants, instead of type use whence (although it may have type aliased to whence -v already as a convenience):

$ ksh --version
  version         sh (AT&T Labs Research) 1993-12-28 p

$ alias type
type='whence -v'

Regards,
Alister

Sorry for my ignorance,
and thanx for informations.

BTW.

% which echo
echo: shell built-in command
% type -a echo
echo is a shell builtin
echo is /bin/echo

Which versions of zsh are you using?

print $ZSH_VERSION

I also had such experience with old zsh versions (on Solaris < 10 default distributions, for example).

Could you please also post the versions and the timings of your tests with the other shells?

zsh version:

4.3.10

ksh version

Version JM 93t+ 2009-05-01

ksh
100000 loops

1.8 sec

ksh
500000 loops

9.4 sec

Yes,
I can reproduce it, it seems to be related to that particular expression: for loop with brace expansion.

It's not the expansion by itself:

% time bash --norc  -c ': {1..300000}'
bash --norc -c ': {1..300000}'  0.76s user 0.54s system 95% cpu 1.357 total

% time zsh -fc ': {1..300000}'        
zsh -fc ': {1..300000}'  0.12s user 0.16s system 96% cpu 0.300 total

It's not the for loop:

 % time bash --norc -c 'for ((;++i<=300000;)); do :;done'
bash --norc -c 'for ((;++i<=300000;)); do :;done'  2.62s user 1.99s system 89% cpu 5.135 total

% time zsh -fc 'for ((;++i<=300000;)); do :;done'
zsh -fc 'for ((;++i<=300000;)); do :;done'  0.83s user 1.94s system 82% cpu 3.345 total

As a workaround I would suggest using another form of the for or while loop.

Raduolov -

I cannot duplicate it in 3.0.8 zsh on Solaris 9. Try truss on the for loop and grep for fork
to see if it is maybe creating children all over the place.

Hi Jim,
could you post the code and the exact timings from your test with 3.0.8. This is what I get:

% uname -r
5.9
% print $ZSH_VERSION
3.0.8
% time zsh -fc 'for i in {1..100000}; do :;done'
zsh -fc for i in {1..100000}; do :;done  14.84s user 0.33s system 99% cpu 15.204 total
% time zsh -fc 'for i in {1..200000}; do :;done'
zsh -fc for i in {1..200000}; do :;done  59.64s user 0.81s system 99% cpu 1:00.76 total

Doesn't seem to scale very well :slight_smile:

If anybody is interested, here is Bart Schaefer's (one of the maintainers) reply.