I need to understand the differences between the bash shell and the Bourne shell

I do not claim to be an expert, but I have done things with scripts that whole teams of folks have said can not be done. Of course they should have said we do not have the intestinal fortitude to git-r-done.

I have been using UNIX actually HPUX since 1992. Unfortunately my old computer died and the proprietary software that I was using can not be used on Win 7. I downloaded cygdrive so that I could continue to run several awk and sed scripts I have written. These worked fine in my old Bourne shell (sh)

1st Question/comment. I have seen that #!/bin/bash must be the first line of a script. I see this as a comment not a command. Seems to me that there is confusion either on my part or the rest of the world.

The previous permissions on my script were lost probably due to windows.

I changed my script "trans_xy" to full permissions

"trans_xy"

#!/bin/bash # I think this command is BS > debug effort
# Process XY data
echo Process XY data
#cp 60-121130-01r2_ipc.net ipc # debug attempt ipc exist
sed s/" -"/" DOT."/ ipc > raw
awk '$2~/MTG/{print $0}' ipc > mtg
awk '$2~/TP/{print $0}' ipc > tp
awk '$3~/DOT/{print $0}' raw > raw1
awk '$2~/VIA/{print $0}' raw1 > vias
awk '$2!~/VIA/{print $0}' raw1 > raw_xy
sed s/"R180"/" R180"/g raw_xy > raw_xy_1
sed s/"R270"/" R270"/g raw_xy_1 > raw_xy_2

chmod 777 trans_xy
ls -l
trans_xy -rwxrwxrwx

the result copied from the bash window intresting font from the stdout

$ ./trans_xy
Process XY data
: No such file or directory
: No such file or directory
: No such file or directory
: No such file or directory
: No such file or directory
: No such file or directory
: No such file or directory_1
: No such file or directory_2

If I paste the commands into bash they work like a champ.
I suspect that bash does not know to do it's work in the pwd it seems odd that the output file is referenced in the crude error message displayed on the stdout.

Notice that the error message is screwed up

command from script
sed s/"R180"/" R180"/g raw_xy > raw_xy_1
result
: No such file or directory_1

command from script
sed s/"R270"/" R270"/g raw_xy_1 > raw_xy_2
result
: No such file or directory_2

Does anyone know if what the heck is happening here?

The line

#!/bin/bash

is the so-called "shebang" line which tells the system what interpreter to run the script in. It's pretty much the same as the magic bytes in executable formats, like ".ELF" (UNIX) or "MZ" (Windows .exe). This was needed after other shells and interpreters came available other than just the Bourne Shell.

Your other errors probably stem from the sed commands, as probably starting with Korn Shell (which is the base for the POSIX standard shell) the commands for sed have to be passed as a single argument, but the shell splits arguments on spaces. So your first command should (as an example) be written as

sed -e 's/ -/ DOT./' ipc > raw

Further to pludi.

The shebang line must be exactly as in pludi's post. The line is only needed if your default shell is not bash and you want to use bash though many use it to make it clear to others how to read the script. (On modern HP-UX the default shell /usr/bin/sh is actually a POSIX shell and there is no bash).

#!/bin/bash

Any extra characters after the above string are passed to the shell as one parameter signifying a filename for the shell to execute - whether or not a hash character is present. A hash character is valid in a filename.
Therfore this lines is invalid and will cause error messages.

#!/bin/bash # I think this command is BS > debug effort

Further to pludi.

Even if user insists on changing exactly what is being said in sed command, it would not make any difference from what you are replying because the command can be written as:

sed -e 's/" -"/" DOT."/' ipc > raw

because the pair of single quotes engulf the double pair of double quotes and thus the pattern match will consider the double quotes as literal.

sed -e is not required if the arguments following the command are valid arguments. But for grins I tried it on the command line it works as a stand alone command in the script file name is not found.

I still think the problem is that the bash script does not know to look in the directory that it is excuted from. In fact the all the files are in the working directory from when I executed the script line by line.

Lets focus on the problem and not the code in the script.

so if I'm only using a bash shell then this is not required?

#!/bin/bash

If I copy all the lines in the script and paste on the bash command line it works. Is the syntax requirement for a script not the same as line by line commands?

---------- Post updated at 05:24 PM ---------- Previous update was at 04:23 PM ----------

what is the default standard output in a bash shell?

echo blah blah

result on screen
blah blah

awk '$2~/MTG/{print $0}' ipc

result on screen
blank screen
in ksh and others the result is directed to the stdout (the Screen)

---------- Post updated at 06:24 PM ---------- Previous update was at 05:24 PM ----------

using either wordpad or notepad as a script editor

simple source file "A"
Apple
Pear
Orange
Grape

original code

#!/bin/bash
echo Process fruit
touch Apple 
awk '$1~/Apple/{print $0}' A #> Apple

result
$ ./trans_xy
Process fruit
touch: cannot touch `Apple\r': No such file or directory
Apple

SOLUTION

#!/bin/bash
echo Process fruit
touch Apple #
awk '$1~/Apple/{print $0}' A > Apple #

result
$ ./trans_xy
Process fruit

file "Apple"
Apple

my suspicion is that Windows 7 has extra hidden garbage "\r" at the end of every line, whats weird is that if I copy and paste the entire script to the command line it works as intended?

Microsoft, since the early days of DOS, uses the two bytes carriage-return/linefeed as end-of-line terminators (and ^Z instead of ^D for EOF). So this isn't a Windows 7 specialty. But when copy-and-pasting lines, the command prompt isn't sent that 2 byte sequence, but an emulation of the enter key, where it's up to the program to handle that correctly. Cygwin comes with both vim and emacs (and a multitude of other editors) that let you save files in UNIX format.

You can leave out the shebang line if you really only use the bash shell. But expect your script to fail as soon as someone tries to run it in another shell (ksh/csh/ash/dash/...). By default, the script is executed using the same shell the user is currently running, but with the shebang line you'll tell the system explicitly which shell to use. This is especially useful if you use different scripting languages:

• You can write stand-alone awk programs using
  text #!/usr/bin/awk -f
  
• Perl programs will almost guaranteed fail if you run them without stating the interpreter somewhere

The same is true for expect, TCL, Python, Ruby, ....

The standard output for bash (just like any other POSIX shell) is stdout. If that is connected to a terminal, output goes there. If it's connected to a file, likewise. Maybe it's just that your awk statement didn't match, and so didn't print anything.

The script worked in true unix shells. I have a copy of my script untouched by DOS and it has the same problem. I understand the CR/LF problem maybe the true unix shells deal with CR/LF wheras bash does not. Is vim some sort of bastardized version of vi?

bash is interpetting the line breaks as a directory\r "\r" is appended to the to the destination file name. Why does it not "Ralph" on the echo staments?

I think it best to not use bash if real shells are available IMHO so I prefer not to force a real shell to use bash.

I guess my work around is to insert # before the CR. that would be compatible with any shell even bash.

I hate to bash bash since I downloaded it for free but I don't think it is as good as real unix shells.

The awk commands work as proven by command line execution using bash and previous scripts run in the korn shell.

In a KISS (Keep It Simple Stupid) script

 
touch Apple

results in
touch: cannot touch `Apple\r': No such file or directory

echo me thinks bash is not so good

result
me thinks bash is not so good(nonprinting characters)

vim is an acronym for vi improved. Works just like ye olde vi, just has some perks added (such as multiple undo and command recall).

As for your problem with the shell script, I can't quite make out the difference between bash and "a real/true" shell (what's the difference for you anyways? A certain version?). On Linux, using bash and AT&T Korn Shell (test.sh was created on Windows XP using Notepad):

> cat test.sh
touch Apple
echo me thinks bash is not so good
> /bin/bash test.sh
touch: cannot touch `Apple\r': No such file or directory
me thinks bash is not so good
> /usr/bin/ksh test.sh
touch: cannot touch `Apple\r': No such file or directory
me thinks bash is not so good

On HP-UX, again using the same shells:

$ /usr/local/bin/bash test.sh
me thinks bash is not so good
$ ls -b Apple*
Apple\015
$ rm Apple*
$ /usr/bin/ksh test.sh
me thinks bash is not so good
$ ls -b Apple*
Apple\015

Both AT&Ts Korn Shell and bash are fully (to my knowledge) POSIX compatible shells, based on the original Korn shell. The original Bourne Shell isn't POSIX compatible, neither is the C Shell (and it's successors).

As for bash being a free shell: it's not the only one. AT&Ts Korn Shell has public available source code since 2000, and most Linux distributions have since added it to their repositories (including Cygwin). And then there's ash, dash, zsh, ...