Shell Script to monitor folder and upload found files via FTP

Hi everyone!

I'm in a need of a shell script that search for all files in a folder, move all those files to a temp folder, and upload those files via FTP. When the file transfer via FTP completes successfully, the file is moved to a completed folder. In case any of those files fails, the file will be moved to a failed folder.

I have very little knowledge on Shell Scripting, but I'll try to explain as clear as possible what I need using a example flow below.

#!/bin/sh

# Configure variables at the begining of the script
MONITOR_DIR=/path/to/monitor/files/
TMP_DIR=/path/to/monitor/files/temp/
SUCCESS_DIR=/path/to/monitor/files/success/
FAILED_DIR=/path/to/monitor/files/failed/
PROCCESS_ID_DIR={execution timestamp string}
FILE_EXT=flv

# FTP variables
FTP_HOST=ftp.myserver.com
FTP_USER=mysername
FTP_PASS=mypassword
FTP_REMOTE_DIR=/

The script will work with these steps:

1. For each file with extension $FILE_EXT in $MONITOR_DIR
2. echo "found file: " and the name(s) of file(s) found
3. Move all found files to $TMP_DIR/$PROCCESS_ID_DIR (folder $PROCCESS_ID_DIR has to be created)
4. For each file in $TMP_DIR/$PROCCESS_ID_DIR
5. Upload via FTP each file individually using the FTP variables
6. If FTP transfer completes, move that file to $SUCCESS_DIR, else move it to $FAILED_DIR
7. echo "file (file_name) transfered" or "file (file_name) failed" depending on what happened above
8. When all the files in $TMP_DIR/$PROCCESS_ID_DIR are processed, delete $PROCCESS_ID_DIR folder in $TMP_DIR
9. quit script

That's basically it. I need this to monitor/backup/copy-to-CDN files uploaded to my site using a cron file.

Any help will be greatly appreciated...

Thanks!

---------- Post updated 07-24-10 at 12:19 AM ---------- Previous update was 07-23-10 at 10:04 AM ----------

Created this script

#!/bin/sh

# Configure variables at the begining of the script
MONITOR_DIR=/var/www/web12/web/php-site-monitor/files
TMP_DIR=/var/www/web12/web/php-site-monitor/files/tmp
SUCCESS_DIR=/var/www/web12/web/php-site-monitor/files/success
FAILED_DIR=/var/www/web12/web/php-site-monitor/files/failed
FILE_EXT=flv

# FTP variables
FTP_HOST=upload-ftp.simplecdn.com
FTP_USER=myuser
FTP_PASS=mypass
FTP_REMOTE_DIR=/

cd $MONITOR_DIR
COUNT_FILES=$(ls -l *.$FILE_EXT | grep ^- | wc -l)

if [ $COUNT_FILES -gt 0 ]; then

echo "A total of" $COUNT_FILES "file(s) found.";
echo "Creating TEMP directory:" $TMP_DIR/$PROCCESS_ID_DIR;

mkdir $TMP_DIR/$PROCCESS_ID_DIR

for f in *.$FILE_EXT; do
echo "Moving file:" $f "to:" $TMP_DIR/$PROCCESS_ID_DIR;
mv $f $TMP_DIR/$PROCCESS_ID_DIR
done

cd $TMP_DIR/$PROCCESS_ID_DIR

for f in *.$FILE_EXT; do
echo "Uploading file via FTP:" $f
ftp -in $FTP_HOST <<EOF
user $FTP_USER $FTP_PASS
binary
cd $FTP_REMOTE_DIR
put $f
bye
EOF

if [ $? -eq 0 ]
then
	echo "FTP File Upload Completed:" $f
	mv -f $f $SUCCESS_DIR
else
	echo "FTP File Upload Failed:" $f
	mv -f $f $FAILED_DIRs
fi

done

echo "Uploads completed...";
echo "Deleting TEMP folder:" $TMP_DIR/$PROCCESS_ID_DIR;

rm -rf $TMP_DIR/$PROCCESS_ID_DIR

else

echo "No files found... Exit!";

fi

It does the work...

Any recommendations?

or...
yum -y install inotify-tools

while true; do inotifywait -r -e MODIFY dir/ && <script>; done;

or

while inotifywait -r -e MODIFY dir/; do make; done;

The script is to be run as a cron in Rackspace Cloud Sites, so that's the only option I have.

Sorry I couldn't be of more help.
Some one with more Kung-Fu will be along to assist.

Have a Great Day!

You to...