Improve sftp script

Shell Programming and Scripting
1

Dear all,

I have written two scripts to transfer files to another server outside the company.
One is a batch script , and the other script calls the batch script, send the files and archive the file sent.

The problem is, that I want to get the list of files which have been uploaded the the other server, in order to be sure that all files have been transferred.

Below is the batch script, and the other script.

Please, would you take a look and give any suggestion to improve the performance of these scripts?

Thank you in advance!!

batch-script:

UploadFile=$1
ServerSftp=$2
echo put $UploadFile > /tmp/batch-sftp$$.log
# Send the file throught SFTP as Batch
sftp -b /tmp/batch-sftp$$.log $ServerSftp
# Remove log file
rm -f /tmp/batch-sftp$$.log

push script:

$data = `date +%c`;
chomp($data);

# get the list of files.
@files = </usr/local/xxx/yyy/zzz/*>;

foreach $file (@files) {
# Send each file with SFTP
$sftp_put = `/usr/local/scripts/batch-sftp.sh "$file" username\@192.168.xx:xx/upload 1> /dev/null; echo \$?`;
# clean out-put from newline
chomp($sftp_put);

        if ($sftp_put eq "0") {
                $move_archive = `mv "$file" /usr/local/archive/ > /dev/null; echo \$?`;
                    chomp($leviz_archive);
                                              if ($move_archive == "0") {
                                $write_logs = `echo \"$data - OK - "$file" was sucessfully uploaded\" >> /usr/local/logs/outgoing.log`;

                                $write_logs = `echo \"$data - INFO - "$file" was sucessfully moved to /usr/local/archive/\" >> /usr/local/logs/outgoing.log`;
                           } else {
                                 $write_logs = `echo \"$data - ERROR - "$file" cannot be moved to /usr/local/archive/ : "$move_archive"\"  >> /usr/local/logs/outgoing.log`;
                        }
        } elsif ($sftp_put ne "0") {
             $write_logs = `echo \"$data - ERROR - "$file" cannot be uploaded : "$sftp_put"\" >> /usr/local/logs/outgoing.log`;
        }
 }
2

Why not use sftp's capabilities like "abort on error" or "raised logging level" ( man sftp )

to make sure all files have been transmitted correctly instead of that cumbersome later analysis? Create the batchfile carefully, and then check the logfile for anomalies.

3

Thank you for replying!
Please, would you help me on editing this script?

4

Is there a particular reason you have a perl script program and a shell script? Could this all be wrapped up into 1 shell script?

5

That's not too easy remotely. Create a test batch file, use

sftp -vb batchfile >logfile 2>&1
echo $?

, do the same with some errors in the batch file (e.g. non-existing files) and then compare/post the logfiles.

6

Dear Rudic, thank you, i will try it.

@Chubler_XL Chubler_XL Thank you. Not, i can wrapped it into 1 shell script.

7

Dear all,

Please any help on this?

8

?

9

How about this shell script:

FILE_DIR=/usr/local/xxx/yyy/zzz
DONE_DIR=/usr/local/archive
BAT_FILE=/tmp/batch-sftp$$.batch
UP_DIR=/upload
LOG_FILE=/usr/local/logs/outgoing.log
SERVER_SFTP=username@192.168.xx:xx

trap 'rm -f $BAT_FILE' 0 1 3 15

cd "$FILE_DIR"

for file in *
do
    [ -f "$file" ] || break
    [ -s $BAT_FILE ] || echo "cd $UP_DIR" > $BAT_FILE
    echo "put \"$file\"" >> $BAT_FILE
    echo "!echo \"Upload of $file done"\
         "- moved to $DONE_DIR/\" >> $LOG_FILE" >> $BAT_FILE
    echo "!mv \"$FILE_DIR/$file\" \"$DONE_DIR/$file\"" >> $BAT_FILE
done

if [ -s $BAT_FILE ]
then
    sftp -b $BAT_FILE $SERVER_SFTP > /dev/null
else
    echo "No files to transfer"
fi
10

Thank you your replies.

If i edit

UploadFile=$1
ServerSftp=$2
echo put $UploadFile > /tmp/batch-sftp$$.log
# Send the file throught SFTP as Batch
sftp -b /tmp/batch-sftp$$.log $ServerSftp
# Remove log file
rm -f /tmp/batch-sftp$$.log

to

UploadFile=$1
ServerSftp=$2
echo put $UploadFile > /tmp/batch-sftp$$.log
# Send the file throught SFTP as Batch
sftp -bv /tmp/batch-sftp$$.log $ServerSftp > /tmp/logfile 
# Remove log file
rm -f /tmp/batch-sftp$$.log

will this save log files to remote server, or to the local server?

Thank you

11

To the local host.