Portable shell script advise

Shell Programming and Scripting
1

All,

I have a need for a portable shell script for LInux and HPUX. The script has a simple need; Check for local files of a specific name (i.e. filename*), scp them to another system, and archive them to a folder. The script runs via cron.

I first wrote the script in Linux (bash with gnu tools) pretty quickly just to get it to function. Then I was asked to do the same type of process running on HPUX (we have 11.31 and 11.23) and did some quick tweaks to get it to run on HPUX. I'm running into issues with the way I wrote it and timing of files written, sent, and archived (I didn't really have the scope of frequency).

Basically I want to put together a portable script with assurances that files will not get 'missed'.

My thought is:

  • do a listing of the desired files when the script is run

    • If no files, exit
    • If files exist, drop the listing into an array

  • use the array to send and archive files

Does this sound like the best/easiest way to accomplish this?

The current problem code I have is:

IFS=!
dest=servername
inst=INST2
logdir=/path/to/log/dir
month=`date +%b%Y`
logfile=${logdir}/xfer_${1}_${month}.log

if [ "$1" = "us" ]; then

        filedir=/path/to/us/dir
        destdir=/path/${inst}/path/us

elif [ "$1" = "eu" ]; then

        filedir=/path/to/eu/dir
        destdir=/path/${inst}/path/eu

fi

# --------------------------------------------------
# functions
# --------------------------------------------------

usage () {

        echo "usage info"

}

header () {

        divider===============================
        divider=$divider$divider$divider

        printf "\n$divider\n" >> $logfile 2>&1
        echo "$1 -- `date`" >> $logfile 2>&1
        printf "$divider\n" >> $logfile 2>&1

}

# --------------------------------------------------
# main
# --------------------------------------------------

if [ $# -ne 1 ]; then

        usage
        exit 1

fi

cd $filedir

        files=`find $filedir -type f -name "iS*" | xargs -n1 printf %f!`

                if [ ${#files[@]} -eq 0 ]; then

                        header "$1 files to send"
                        echo "\n No $1 files to transfer" >> $logfile
                        exit 1

                else

                        header "$1 files to send"
                        ls -l iS* | awk '{print $9}' >> $logfile

                        header "Sending $1 files"
                        cd $filedir; scp iS* $dest:$destdir >> $logfile 2>&1

                        if [ $? -ne 0 ]; then

                                echo "\n  There are no files to transfer..." >> $logfile 2>&1
                                header "Complete"
                                exit 1

                        else

                                header "$1 files successfully sent"

                                header "Moving files and purge 30+ days"
                                mv iS* ${filedir}/archive >> $logfile 2>&1
                                find ${filedir}/archive -type f -mtime +30 -ls -exec rm -f -- {} \; >> $logfile 2>&1
                                header "Finished moving files.."

                        fi

                fi

exit

I know this code is not written very well as there is no checks on archiving only the files that were sent.

Any advise on improving for portability and simplifying is greatly appreciated.

TIA,

Herb

2

echo "\n" isn't portable. Neither is the echo -e some versions of echo require to make \n work. If you want escapes like \n in the string, you must use printf.

Note that it's better to use %s in a printf command string than to substitute variables into it, since you don't want to put more escape sequences into a printf string by accident.

STR="This should print percent signs like %s without bombing or mangling them"
# safe
printf "\n%s\n" "$STR"
# unsafe
printf "\n$STR\n"
3

Here is one that should be safe, efficient, and portable even to older Solaris and HP-UX.

#!/bin/sh

# we want to find programs here, not in an inherited PATH
PATH=/bin:/usr/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/local/sbin
export PATH

dest=servername
inst=INST2
month=`date +%b%Y`
logdir=/path/to/log/dir
logfile=${logdir}/xfer_${1}_${month}.log
fmask="iS*"

# open once with descriptor 4
exec 4>"$logfile"
find 
#exec 4>>"$logfile"
# writing to descriptor 4 will append to the open file

case $1 in

us)
    filedir=/path/to/us/dir
    destdir=/path/${inst}/path/us
;;
eu)
    filedir=/path/to/eu/dir
    destdir=/path/${inst}/path/eu
;;
*)
    echo "unexpected argument: $1"
    exit 1
;;

esac

# --------------------------------------------------
# functions
# --------------------------------------------------

usage () {

    echo "usage info"

}

header () {

    divider="=============================="
    divider=$divider$divider$divider

    printf "\n$divider\n" >&4
    echo "$1 -- `date`" >&4
    printf "$divider\n" >&4

}

# --------------------------------------------------
# main
# --------------------------------------------------

if [ $# -ne 1 ]; then

    usage
    exit 1

fi

# make sure we can go there
if cd $filedir
then

    files=`ls -- $fmask 2>/dev/null`
    header "$1 files to send"

    if [ -z "$files" ]; then

        printf "\n No $1 files to transfer" >&4

    else

        # a lazy printf "$files\n" or echo "$files" is at risk
        printf "%s\n" "$files" >&4

        header "Sending $1 files"
        # a trailing / makes sure it is a directory
        scp -- $fmask ${dest}:${destdir}/ >&4 2>&1

        if [ $? -ne 0 ]; then

            printf "\n  There are no files to transfer..." >&4
            header "Complete"

        else

            header "$1 files successfully sent"

            header "Moving files and purge 30+ days"
            mv -f -- $fmask ${filedir}/archive/ >&4 2>&1
# + collects arguments and calls the program very few times -> faster; no -- needed in find -exec prog {}
# HP-UX does not have -ls
            find ${filedir}/archive/ -type f -mtime +30 -exec ls -l {} + -exec rm -f {} + >&4 2>&1
            header "Finished moving files.."

        fi

    fi

fi

# explicit close of logfile not needed before an exit or end of file
#exec 4>&-

exit
4

@Corona688, @MadeInGermany thank you for your responses..

@MadeInGermany - Quick question:

Regarding the code you post, does the way $fmask is used solve the issue I have been seeing where:

use $fmask to scp files to $destdir

 - between scp and archive of files a new file is written to $filedir

use $fmask to archive files

In this scenario, would any files that were written to $filedir between scp and archive get archived without being sent to $destdir?

Thanks again for any guidance.

Herb

5

No, because I tried to keep the functionality.
The $fmask simply allows to specify the "iS*" once at the beginning of the script. The shell will re-evaluate it every time.

But it is no big deal to replace the $fmask by $files for the scp and mv commands.
Then $fmask is evaluated once with the ls command, where the result goes to $files - and that is static.