I see lot of ad-hoc shell scripts in our servers which don't have a shebang at the beginning .
Does this mean that it will run on any shell ?
Is it a good practice to create scripts (even ad-hoc ones) without shebang ?
They will run in the shell of the user who executed it.
So if user uses /bin/bash, the script will be executed in that shell.
No, that does not mean it will work in every shell, it probably won't.
That's incorrect. When an executable file's header/magic isn't recognized, it is passed to /bin/sh regardless of the user's shell.
Note that this behavior (falling back on /bin/sh ) is only available through some library functions (execlp, execvp, execvpe) which wrap the execve(2) system call. Any code that tries to execute such a script using other means (such as directly invoking execve(2)) will fail.
Regards,
Alister
9 Likes
Note that if the script is sourced (e.g. . /path/to/script.sh ) rather than run as an executable file then it will be run in the current shell.
1 Like
Is that correct?
I'm on FreeBSD (sorry don't know how to get version info). The login uses bash.
If I run a script containing only this line
[[ 1 == 1 ]] && echo yes || echo no
It results in (seems to use bash)
yes
If I enter a sh-shell with sh and run the script I get (seems to use sh)
[[: not found
no
True.
With regard to the original post, if the script is sourced, even if the shebang is present, it is never used.
Regards,
Alister
---------- Post updated at 10:02 AM ---------- Previous update was at 09:39 AM ----------
It should be correct. It's a fundamental and standardized behavior. From your description, it sounds like you are explicitly passing the script to different shells, but I can't be certain. You did not clearly and unambiguously state the exact commands that you used to create your script and run it. Please do so.
For version info, you should be able to get it from uname -a .
Regards,
Alister
---------- Post updated at 10:24 AM ---------- Previous update was at 10:02 AM ----------
The following excerpts are from the current version of FreeBSD's libc. The most relevant bits have been highlighted:
From FreeBSD CVS - src/lib/libc/gen/exec.c
retry: (void)_execve(bp, argv, envp);
switch (errno) {
case E2BIG:
goto done;
case ELOOP:
case ENAMETOOLONG:
case ENOENT:
break;
case ENOEXEC:
for (cnt = 0; argv[cnt]; ++cnt)
;
memp = alloca((cnt + 2) * sizeof(char *));
if (memp == NULL) {
/* errno = ENOMEM; XXX override ENOEXEC? */
goto done;
}
memp[0] = "sh";
memp[1] = bp;
bcopy(argv + 1, memp + 2, cnt * sizeof(char *));
(void)_execve(_PATH_BSHELL,
__DECONST(char **, memp), envp);
goto done;
execvpe(3) calls execve(2) (through a libc wrapper). If the system call fails with ENOEXEC, that means that an executable file was found but its format is not recognized. This happens if the file is a foreign binary or if it's a shell script without a shebang. execvpe(3) then reattempts to execve a bourne shell with the unrecognized file as its first argument.
From FreeBSD CVS - src/include/paths.h
#define _PATH_BSHELL "/bin/sh"
The path to the bourne shell is hardcoded in a macro.
I also took a look at glibc and they too share the same implementation design. execlp and execvp wrap execvpe. execvpe takes care of checking errno after execve, retrying with /bin/sh if appropriate. They both use _PATH_BSHELL to point to /bin/sh.
For glibc, the relevant files:
sourceware.org Git - glibc.git/blob - posix/execvpe.c
sourceware.org Git - glibc.git/blob - sysdeps/unix/sysv/linux/paths.h
Regards,
Alister
9 Likes
Thank you Alister. Thanks everyone.
I have 4 quick questions:
Question 1.
According to Wikipedia /bin/sh means Bourne shell . Is that Right?
Shebang (Unix) - Wikipedia, the free encyclopedia
Question2.
So, in Solaris 10, RHEL Version5, version 6, AIX version 6, 7 , if i forget to add shebang to a script , will the script be executed using Bourne shell ?
Question3.
Regarding CarloM's post on sourcing . /path/to/script.sh
I have used sourcing only to set environmental variables in the current shell. Can sourcing be used to execute shell scripts as well?
Question4.
I have noticed that some people execute shell scripts by putting sh at the beginning like
sh /path/to/myscript.sh
I always execute shell scripts without the sh at the beginning.
/path/to/script.sh
Which is recommended ?
Yes.
Yes. Anything calling itself UNIX will have a Bourne shell available by default. Even on systems where c-shell is popular, Bourne is still available, because too many system things depend on it to exclude it. You could get more than you asked for -- BASH or DASH on Linux for instance -- but you will at least get generic Bourne features.
On Solaris you might get an extremely old pre-POSIX Bourne, to the point it doesn't understand $(this) syntax, only `backticks`, so if you forget the shebang, you might not get everything you need.
Yes, entire shell scripts can be run. You could make an environment script that sets different variables for different users, for instance. Or an entire script of any purpose.
Note that the effect of exit is to make the shell quit. If it's running in a separate shell, it makes that shell quit. If it's running in your shell, it makes your shell quit. 
If you want a sourced script to exit without killing your terminal, use return instead.
Like said above, sh scriptname ignores the shebang, forcing it to use sh no matter what. What if it actually needs a full-fledged BASH or KSH shell, or something weirder like csh? sh scriptname wouldn't work. So I consider it best to let the shebang decide which interpreter to use.
This is a point of debate, however. There's arguments both ways.
1 Like
Here is a screen dump when testing different cases - version info as first command (only some info hidden and added comments as "# ..."):
# new fresh login...
[user@server ~]$ uname -a
FreeBSD xxx.yyy.zz 6.2-RELEASE FreeBSD 6.2-RELEASE #0: Fri Jan 12 11:05:30 UTC 2007 ...
[user@server ~]$ cd test/
[user@server ~/test]$ cat tfs
[[ 1 == 1 ]] && echo yes || echo no
[user@server ~/scripts]$ ls -l tfs
-rwxr--r-- 1 user user 36 Aug 31 20:34 tfs
# run the script (the folder is not in PATH):
[user@server ~/test]$ ./tfs
yes
# testing forcing sh:
[user@server ~/test]$ sh tfs
[[: not found
no
# testing forcing bash:
[user@server ~/test]$ bash tfs
yes
# enter sh:
[user@server ~/test]$ sh
[2 user@server /usr/home/user/test]$ ./tfs
[[: not found
no
# enter bash:
[2 user@server /usr/home/user/test]$ bash
[3 user@server ~/test]$ ./tfs
yes
[3 user@server ~/test]$ exit
exit
[2 user@server /usr/home/user/test]$ exit
[user@server ~/test]$ env
# some taken away...
DL_IMP=../data/import/contacts/
SHELL=/usr/local/bin/bash
TERM=xterm
SSH_TTY=/dev/ttyp1
USER=user
ENV=/home/user/.shrc
PAGER=more
FTP_PASSIVE_MODE=YES
EDITOR=vi
HOME=/home/user
LOGNAME=user
_=/usr/bin/env
...
[user@server ~/test]$
Can it be that it's only when the file is a foreign binary that it falls back to _PATH_BSHELL?
eneric Bourne doesn't have to support [[ ]]. Some do, some don't, so if you don't know what shell you're going to get, limit yourself to [ ] .
You should also have a blank line as the first line of the script, in case your first line accidentally ends up being a command. [ is a command for instance.
So, really, I reccomend having a shebang even if you don't plan on using it, just in case someone else does, and to at least document which variety of shell is preferred for it.
#!/bin/sh
[ 1 -eq 1 ] && echo yes || echo no
@Corona688, thanks for the answer, but the script was only for testing what shell is used when not using shebang - as discussed in this thread.
No. The code in the libc functions (and the kernel syscall they invoke) does not make that distinction.
I can replicate your observation on an old debian system using bash 3.1.17. Bash must not be using any of those libc functions. If it were, /bin/sh would have been called. Bash is sidestepping them, either by invoking execve(2) directly or by using an exec*(3) variant which does not have any of the /bin/sh fallback semantics, so that it can use itself to interpret executable files that aren't recognized by the kernel.
This is allowed by POSIX shell command search and execution, but it is not required.
Like Corona688, I too think it's best to use the shebang. At the very least, it serves as documentation of the author's intentions. At the very most, it's a weak guarantee that the script will be interpreted by a compatible interpreter. Just because it's there, does not mean that it will be used. If the script is sourced by an interpreter, or fed on standard input, or passed as a command line argument -- in other words, when a shell is reading the script -- the shebang is treated as a comment and ignored. The shebang is only relevant when the kernel itself, in execve(2), is reading the script.
Regards,
Alister
@kraljic
To answer your main theme:
It is recommended to specify the full path when executing a script and have a correct Shebang line in the script. This is unambiguous and quick.
It is not recommended to invoke the Shell directly with a parameter of the script name (with or without the full path).
If there is no Shebang line and the Shell has not been invoked directly all unix/Linux systems that I have encountered invoke /bin/sh by default.
On many (by by no means all) modern systems /bin/sh is the Posix Shell (or /bin/bash on Linux).
Your system appears to be behaving like Sun/Oracle Solaris (or many unix O/S derived from Berkeley unix) where /bin/sh is the Bourne Shell. On RedHat Linux /bin/sh is linked to /bin/bash .
Just to verify whether there is a lnk on your system, what is the output from:
ls -lisad /bin/sh
ls -lisad /bin/bash
No, It is wrong. If you agree that it is wrong, just change it.
On modern HP-UX for example /usr/old/bin/sh is the Bourne Shell and /bin/sh is the Posix Shell.
1 Like
This is definitely a shell-implementation detail that boils down to whether the shell is using the libc functions that fallback to /bin/sh or not.
I installed a few shells on someone's Ubuntu 12.04 LTS box and ran a few tests. The results do not in anyway conclusively prove anything about the implementation details except that some of these shells are definitely not using execlp/execvp/execvpe. sh is a symlink to dash. The sh/dash error message is used to determine whether it is used by the other shells.
$ cat tests/shebang.sh
[[ " " ]]
echo $?
$ ls -l /bin/sh
lrwxrwxrwx 1 root root 4 Aug 27 15:15 /bin/sh -> dash
# dash 0.5.7-2ubuntu2 (baseline)
$ sh ./tests/shebang.sh
./tests/shebang.sh: 1: ./tests/shebang.sh: [[: not found
127
$ dash ./tests/shebang.sh
./tests/shebang.sh: 1: ./tests/shebang.sh: [[: not found
127
$ bash --version | head -n1
GNU bash, version 4.2.24(1)-release (x86_64-pc-linux-gnu)
$ bash
$ ./tests/shebang.sh
0
$ bash ./tests/shebang.sh
0
$ ksh --version
version sh (AT&T Research) 93u 2011-02-08
$ ksh
$ ./tests/shebang.sh
0
$ ksh ./tests/shebang.sh
0
$ pdksh
$ print $KSH_VERSION
@(#)PD KSH v5.2.14 99/07/13.2
$ ./tests/shebang.sh
./tests/shebang.sh: 1: ./tests/shebang.sh: [[: not found
127
$ pdksh ./tests/shebang.sh
./tests/shebang.sh[1]: syntax error: `" "' missing expression operator
$ zsh --version
zsh 4.3.17 (x86_64-unknown-linux-gnu)
$ zsh
$ ./tests/shebang.sh
./tests/shebang.sh: 1: ./tests/shebang.sh: [[: not found
127
$ zsh ./tests/shebang.sh
./tests/shebang.sh:1: parse error near `]]'
In summary, bash and ksh use themselves when the kernel does not recognize the executable, while pdksh and zsh use /bin/sh (most likely through libc functions).
Regards,
Alister
1 Like
Thank you Methyl. Thanks everyone.
It is not recommended to invoke the Shell directly with a parameter of the script name (with or without the full path).
Didn't quite get you methyl. Can you elaborate ?
Yes. It is solaris 10 (x86-64 bit) as you've guessed.
$ uname -a
SunOS spikey213 5.10 Generic_147441-01 i86pc i386 i86pc
Output of
ls -lisad /bin/sh
ls -lisad /bin/bash
$ ls -lisad /bin/sh
65 2 lrwxrwxrwx 1 root root 13 Jun 23 00:07 /bin/sh -> ../../sbin/sh
$ ls -lisad /bin/bash
4892 1296 -r-xr-xr-x 1 root bin 654504 Feb 2 2011 /bin/bash
So, in solaris the default
man page of sh doesn't say anything explicitly about Bourne shell. But the word Bourne Shell is mentioned once (shown in red below)
$ man sh
Reformatting page. Please Wait... done
User Commands sh(1)
NAME
sh, jsh - standard and job control shell and command inter-
preter
SYNOPSIS
/usr/bin/sh [-acefhiknprstuvx] [argument]...
/usr/xpg4/bin/sh [+ abCefhikmnoprstuvx]
[+ o option]... [-c string] [arg]...
/usr/bin/jsh [-acefhiknprstuvx] [argument]...
DESCRIPTION
The /usr/bin/sh utility is a command programming language
that executes commands read from a terminal or a file.
----output snipped
.
.
.
The Bourne shell has a limitation on the effective UID for a
process. If this UID is less than 100 (and not equal to the
real UID of the process), then the UID is reset to the real
.
.
.
I ran some simple tests on OSX and it appears that when the shebang is not specified, the first sh is used in the search path, rather than /bin/sh
That's either a shell implementation detail (the shell is not using a libc exec?p() interface) or Apple's libc does not adhere to its documentation.
From Apple's libc exec* manual (emphasis mine):
Regards,
Alister