help with scripting

csaunders · August 18, 2005, 8:43am

Need help on how to script this

Login to your host; root access is not required.
cd to the directory /backup/exp.
ls -lrt the directory. There should be tar files for the past two days; one of which should be for today. If files exist that are older than four days or today's tar file is missing, report this on the checklist for further investigation

Im a complete noob, so any help would be appreciated. basically I want a success, that the tar files are present, or a failure, that they are not present.

As well as delete old files, i.e.

Login to your host; root access is not required.
cd to the directory /backup/exp
ls -lrt the directory. There should be a .tar file with today's date and one with yesterday's date. Use the rm command to remove .tar files in that directory that have last months date.

vertigo23 · August 18, 2005, 1:37pm

Is the script going to be running on a different host? That is, is step 1 (log onto the host) part of the script, or do you log in, and then run the script locally?

If you need to do a remote login from inside of the script, your best bet might be to use Expect, as it can literally simulate a manual user session. Syntax is IMHO awful though.

csaunders · August 22, 2005, 11:35am

this is assuming the admin is on the machine and logged in, no need to remote login, so you run the script locally

bakunin · August 23, 2005, 8:08am

Please provide more information if the solution presented afterwards doesn't fit your needs.

You haven't told us how you identify the tar files you are looking for: do they have a naming convention like "mytarDDMMYYYY.tar" or can they have any name and you have to use the creation/modification timestamp from the inode to find out which one is from today?

The following solution assumes the files do have a naming convention in the form mentioned above.

Notice (we will enjoy getting you started but will not continue to write the scripts for you, so you should learn to write scripts yourself) that a script basically consists of commands you would use on the commandline too. Everything you write on the command line you can write into a file, make that file executable and you have a script.

Two things are different though: on the commandline you get some output and upon this you decide how to proceed. In a script you have to provide either decision in advance and take care about what the script should do. This is called "conditional" execution of statements. For instance: everyday you log in to the machine, change to the directory and look for a specific file. If it is there you copy it to somewhere else, if it isn't you have to call your colleague because he forgot to create it. The script of course can't call your colleague, but could mail him an alert. This could look like:

if [ -e /foo/bar/file.to.be.expected ] ; then         # when the file is there
     cp /foo/bar/file.to.be.expected /somewhere/else  # then copy it
else                                                  # but if it isn't there
     MailMyColleague                                  # then notify the lazy SOB
fi

The two "branches" of the if-statement represent the two possible courses of action, the condition (between the rectangular brackets) is the condition you too would use to decide which way to take, just stated in a formalized manner.

The second thing which is different to the command line is the usage of variables. Variables hold values which could change during the various runs of the script but require the same logic (the aforementioned decision process provided in advance) to be handled. Suppose you do not have to look at one directory and check that for files, but to several. The logic is always the same: if a certain file is there, then copy it, else mail an alert. Still, the name of the directories and maybe the name of the files may change from one run of the script to the next. Instead of writing a new script for every occasion we would write one script and either give it the appropriate values as parameters or let it calculate them by itself.

After this rather lengthy (and very incomplete) introduction to programming in general and script programming in specific the promised program:

#!/bin/ksh

typeset chDate="$(date +'%d%m%Y')"
typeset iErrLvl=0

if [ -e /backup/exp/mytar${chDate}.tar ] ; then
     print - "Everything ok, mytar${chDate}.tar is there"
else
     print -u2 "Error: mytar${chDate}.tar is missing"
      (( iErrLvl = 1 ))
fi

exit $iErrLvl

bakunin

csaunders · August 23, 2005, 10:38am

thank you!
Well the format of the file is giving me problems now, here is the format

AIDMS_usvs-master050823061500.tar.Z

so I wrote a script based on yours

#!/bin/ksh
export HNAME=`hostname`
echo $HNAME
export STAMP=`date +"%y%m%d"`

typeset TAR_FILE="AIDMS_"$HNAME$STAMP*
typeset iErrLvl=0

if [ -e /backup/exp/$TAR_FILE ] ; then
print - "Everything ok, $TAR_FILE is there"
else
print -u2 "Error: $TAR_FILE is missing"
(( iErrLvl = 1 ))
fi

exit $iErrLvl

The star is meant as a wildcard, because obvisouly I cant use second, minute, hour that is in the original file format, so Im not sure how to only search for the file name without the second, minute and hour appended to the end...

bakunin · August 24, 2005, 9:23am

Very well done for a first attempt.

You missed your goal, but only by a small margin, here is why, along with some general remarks:

You should not use backticks (`...`), the modern standard is to use the "$(...)" instead. Many still use the old format, but it is not recommended.

The asterisk ("") is a special character to the shell. To use it as a literal character you have to protect it from being interpreted. Usually this is done by surrounding it with quotes: '' or "". It is always a good idea to play it safe in scripts and quote as much as possible. Double quotes protect asterisks and similar chars (wildcards), but do NOT protect dollar signs. Hence, instead of 'typeset filename="ABC"$DATE...' you could surround everything with quotes and write 'typeset filename="ABC${DATE}*"'.

Further, when concatenating such string variables, it is a good idea to use the complete syntax for variables: "$var" is just an abbreviation for "${var}". The former is ok as long as it is (potentially) unambiguous, but the latter is always accurate. Hence, to avoid confusion of the shell (especially when using wildcards) "ABC${DATE}" is better than "ABC$DATE".

At last some personal tips: scripts tend to start out small and become bigger and bigger over time. To write bigger scripts means nothing short of DEVELOPING SOFTWARE and therefore it is a good idea to learn from the software engineers practices - these have basically been developed to help them stay organized in their tons of code. What fits them in their large scale should do for us in our somewhat smaller scale too. It goes beyond the scope of this thread to start a discussion about coding practices (although that would be very interesting), here is just my "personal starter kit of do's and don't's to make life easier":

Always declare your variables. This is not necessary, but a good way to not only keep track of them but also a chance to add a comment about what the variable contains.

Out of personal habit i use prefixes to determine the datatype of a variable: "f" for filenames/directories, "l" for boolean (logical) values, "ch" for characters, "i" for integers. (In fact I use a simplified form of "Hungarian Style Notation".) Of course a shell doesn't know about datatypes and it is possible to use an int value as string, but as soon as you try "if [ $var -gt 100 ]" on a character variable you'd get an error. Staying clear of such confusion avoids these problems.

Talking about variables names: make them speaking! It is a lot easier to type a few characters more than it is to rember what "pidblftx1" is standing for. It is easier to understand the line "if [ $iNextUserID -gt $iMaximumUserID ]" than it would be to read "if [ $nuid -gt $muid ]". More likely than not you will soon forget what "$a", "$A" and "$B" contain, a speaking name is a good way on saving time you would otherwise have to spend on documentation.

Always comment your code. Try to picture yourself in a year from now, having long forgotten what you did today and try to write down for that person what you are doing. I have three standard commenting places: every variable declaration gets a comment telling its content: "typeset -i iUserCnt=0 # counter for user accounts". The next place is the header, before any code starts, with an explanation about what the script does and if it does it in a counterintuitive way why this is so. The third place for comments are code blocks: every group of statements belonging together is commented with a short explanation about what the block does, something like "derive home directory from UID" or so. This makes it easier to navigate between functional entities in the script.

Here is your revised script, it should do now what you want it to do:

#!/bin/ksh
# search for tar files in /backup/exp and print message

typeset chHostName="$(hostname)"                 # the hosts name
typeset chTimeStamp="$(date +"%y%m%d")"          # todays date
typeset fName=""                                 # file name buffer
typeset iErrLvl=0                                # exit code: 0=ok
                                                 # >0: no tar files

fName="AIDMS_${chHostName}${chTimeStamp}*tar.Z"  # put filename together

print - "$0 checking for $fName on $chHostName"

if [ -e /backup/exp/$fName ] ; then
     print - "Everything ok, $fName is there"
else
     print -u2 "Error: $fName is missing"
     (( iErrLvl = 1 ))
fi

exit $iErrLvl

Sorry for this longwinded post, I'm a Mullah in the Sect of Clean Coding and enjoy it deeply to preach about the subject. ;-))

bakunin

csaunders · August 24, 2005, 1:44pm

bakunin:

Very well done for a first attempt.

You missed your goal, but only by a small margin, here is why, along with some general remarks:

You should not use backticks (`...`), the modern standard is to use the "$(...)" instead. Many still use the old format, but it is not recommended.

The asterisk ("") is a special character to the shell. To use it as a literal character you have to protect it from being interpreted. Usually this is done by surrounding it with quotes: '' or "". It is always a good idea to play it safe in scripts and quote as much as possible. Double quotes protect asterisks and similar chars (wildcards), but do NOT protect dollar signs. Hence, instead of 'typeset filename="ABC"$DATE...' you could surround everything with quotes and write 'typeset filename="ABC${DATE}*"'.

Further, when concatenating such string variables, it is a good idea to use the complete syntax for variables: "$var" is just an abbreviation for "${var}". The former is ok as long as it is (potentially) unambiguous, but the latter is always accurate. Hence, to avoid confusion of the shell (especially when using wildcards) "ABC${DATE}" is better than "ABC$DATE".

At last some personal tips: scripts tend to start out small and become bigger and bigger over time. To write bigger scripts means nothing short of DEVELOPING SOFTWARE and therefore it is a good idea to learn from the software engineers practices - these have basically been developed to help them stay organized in their tons of code. What fits them in their large scale should do for us in our somewhat smaller scale too. It goes beyond the scope of this thread to start a discussion about coding practices (although that would be very interesting), here is just my "personal starter kit of do's and don't's to make life easier":

Always declare your variables. This is not necessary, but a good way to not only keep track of them but also a chance to add a comment about what the variable contains.

Out of personal habit i use prefixes to determine the datatype of a variable: "f" for filenames/directories, "l" for boolean (logical) values, "ch" for characters, "i" for integers. (In fact I use a simplified form of "Hungarian Style Notation".) Of course a shell doesn't know about datatypes and it is possible to use an int value as string, but as soon as you try "if [ $var -gt 100 ]" on a character variable you'd get an error. Staying clear of such confusion avoids these problems.

Talking about variables names: make them speaking! It is a lot easier to type a few characters more than it is to rember what "pidblftx1" is standing for. It is easier to understand the line "if [ $iNextUserID -gt $iMaximumUserID ]" than it would be to read "if [ $nuid -gt $muid ]". More likely than not you will soon forget what "$a", "$A" and "$B" contain, a speaking name is a good way on saving time you would otherwise have to spend on documentation.

Always comment your code. Try to picture yourself in a year from now, having long forgotten what you did today and try to write down for that person what you are doing. I have three standard commenting places: every variable declaration gets a comment telling its content: "typeset -i iUserCnt=0 # counter for user accounts". The next place is the header, before any code starts, with an explanation about what the script does and if it does it in a counterintuitive way why this is so. The third place for comments are code blocks: every group of statements belonging together is commented with a short explanation about what the block does, something like "derive home directory from UID" or so. This makes it easier to navigate between functional entities in the script.

Here is your revised script, it should do now what you want it to do:
#!/bin/ksh
# search for tar files in /backup/exp and print message

typeset chHostName="$(hostname)"                 # the hosts name
typeset chTimeStamp="$(date +"%y%m%d")"          # todays date
typeset fName=""                                 # file name buffer
typeset iErrLvl=0                                # exit code: 0=ok
   # >0: no tar files

fName="AIDMS_${chHostName}${chTimeStamp}*tar.Z"  # put filename together

print - "$0 checking for $fName on $chHostName"

if [ -e /backup/exp/$fName ] ; then
   print - "Everything ok, $fName is there"
else
   print -u2 "Error: $fName is missing"
   (( iErrLvl = 1 ))
fi

exit $iErrLvl
Sorry for this longwinded post, I'm a Mullah in the Sect of Clean Coding and enjoy it deeply to preach about the subject. ;-))

bakunin

I apprecaite your help, and thanks for the kind words. Works like a charm, you da man!!!(possibly) or woman!!!

Something more I would like to add is too look at the same file, but since yesterdays date? Any ideas?