FTP Script Help!!

Don_Cragun · November 5, 2014, 2:20am

aia:

@No Don, I did not

This is what I said:

I would like to bring to your attention the following combination of commands in red, which has the potential of spread disaster.

The command cd will silently change directory to your $HOME if it can not use $READDIR to change. Once's in there I don't know the mayhem that it would do, but for sure you can kiss goodbye to your data in $HOME.

Noticed that I only posted a small portion of the code to just bring attention to the possible harmful practice, tenting fate with rm * . I did not even do reference to the specific code, except for the $READDIR because it was there. I could had said cd $NOTHING_HERE and it would had done the job. That portion of the code was copied to help the OP know what I was talking about.

You described the specific condition I pointed out in your point 1:

But any of the others, would nicely had created the same disaster.

With the code being discussed in this thread:

...
export READDIR=/home/foobaruser/TDE
#export TEMP=/opt/foobar/$ENV/tmp
export TDE_BACKUP=/home/foobaruser/TDE/backup

cd $READDIR
...

as I said before, there is absolutely no way that the cd command shown in red above is going to "silently change directory to your $HOME if it can not use $READDIR to change". I said there were three ways that a cd command could move you to $HOME AND NONE of them are possible with this code.

I fully agree that ignoring exit codes from cd , cp , sftp , and other utilities is a recipe for disaster; but there is no way the above code is ever going to silently change directory to $HOME .

Chubler_XL · November 5, 2014, 3:02pm

I have seen some scripts that put the file over with a temporary name then rename. This also protects from jobs on the destination server attempting to process file still in transit.

Once the remote rename is done the file can then be moved to a transfered folder, for eaxample:

put ORD049382.20141106 tmp_transfer
rename tmp_transfer ORD49382.20141106
!mv ORD049382.20141106 tmp /var/orders/transfered/

Here is an attempt at putting Don Cragun's suggestions and this change all together for you:

#!/bin/bash
source /opt/foobar/$1/config/usrconfig.sh
source  /opt/foobar/$ENV/config/$2

### foobar environment variables ###
export FTPHOST="sftpserv.foobar.com"                 # server to ftp to
export FTPUSR="usmssh"                               # ftp user id
export FTPPASS=""                                    # ftp user password
export PATTERN=                                      # pattern for file name to ftp
export USING_MARKER=no                               # using a marker file???
#export TDEMARKER="X"                                # pattern for marker file if no files to ftp
export FTPDIR="prod/TDE/in/"  # directory to ftp to on ftp server
export DAEMONLOG=/opt/foobar/$ENV/log/TDEdeamon.log
export DAEMONSLEEP=10
export CURRDATE=`date +%Y%m%d`
export READDIR=/opt/foobar/$ENV/data/TDE/output
export TEMP=/opt/foobar/$ENV/tmp
export TDE_BACKUP=$APP_ROOT/data/TDE/backup/

if [ ! -d "$READDIR" ]
then
    echo "Directory $READDIR not found!" >>$DAEMONLOG 2>&1
    exit 1
fi

cd $READDIR
sleep 2

> tde_batch
find $READDIR -type f -name '*.ready' | while read file
do
  real_file="${file%.ready}"
  echo "put \"$real_file\" \"$FTPDIR/trans.temp\"" >> tde_batch
  echo "rename \"$FTPDIR/trans.temp\" \"$FTPDIR/$real_file\"" >> tde_batch
  echo "!mv \"$real_file\" \"$TDE_BACKUP/\"" >> tde_batch
done

if [ ! -s ./tde_batch ]
then
     echo "No ready files found in $READDIR" >>$DAEMONLOG 2>&1
     exit 2
fi

echo "quit" >> tde_batch
sftp -b tde_batch "$FTPUSR@$FTPHOST" >>$DAEMONLOG 2>&1
mv tde_batch "$TDE_BACKUP/tde_batch.$CURRDATE"
exit 0

Aia · November 5, 2014, 10:11pm

don cragun:

With the code being discussed in this thread:
...
export READDIR=/home/foobaruser/TDE
#export TEMP=/opt/foobar/$ENV/tmp
export TDE_BACKUP=/home/foobaruser/TDE/backup

cd $READDIR
...
as I said before, there is absolutely no way that the cd command shown in red above is going to "silently change directory to your $HOME if it can not use $READDIR to change". I said there were three ways that a cd command could move you to $HOME AND NONE of them are possible with this code.

I fully agree that ignoring exit codes from cd , cp , sftp , and other utilities is a recipe for disaster; but there is no way the above code is ever going to silently change directory to $HOME .

I don't see that first red highlighted code in my post, in question. Maybe there was some reason for not being included.

gkelly1117:

cd $READDIR
find $READDIR -type f -name '*.ready' | while read file
do
real_file="${file%.ready}"
echo "put $real_file $FTPDIR" >> tde_batch
cp $real_file $TDE_BACKUP
done
echo "quit" >> tde_batch
sftp -b tde_batch "$FTPUSR@$FTPHOST"
cp tde_batch $TDE_BACKUP/tde_batch.$CURRDATE
rm tde_batch
rm *
exit 0[/CODE]

I would like to bring to your attention the following combination of commands in red, which has the potential of spread disaster.
In a script, cd into a directory and accepting it blindly, is very dangerous, for example:

The command cd will silently change directory to your $HOME if it can not use $READDIR to change due to being unset or empty. Once's in there I don't know the mayhem that it would do, but for sure you can kiss goodbye to your data in $HOME.

Some ways to minimize the risk is to use absolute paths and if you cd check $PWD or pwd, and compare with where you should be.

Furthermore, the content of the directory where you are executing the script from is at risk of being destroy or damaged, if $READDIR is set to a path that can not change into. Success of
if [ -d $READDIR ]
is not indication that you can cd into it, even if the directory exist.

In order to further clarify my intent at the first try, I would like to amend my original text with added blue.