How to make sure a file is uploaded in its entirety

Shell Programming and Scripting
1

Hi, I have a client who uploads files to my FTP server on a regular basis. Instead of checking the server all the time, I wrote a script that e-mail me to let me know that there is a new file and attaches is it in the e-mail (they're all small compressed archives). The potential issue with the following script is that it will move a file to the processed directory even if it's not fully downloaded. I'm scheduling the script to run every hour on the hour but if the client is in the middle of uploading a file when it runs, I may only get a part of it in the e-mail. Is there a way to make sure that the files isn't being modified before moving it?

sdir="/home/sftp/"
fc=`ls $sdir |wc -l`
fn=`ls -t $sdir |head -1`
if [ "$fc" -ne 0 ];then
        while [ "$fc" -ne 0 ]
                do
                        ts=`date +%Y%m%d_%H%M%S`
                        ( echo "New file attached"
                                uuencode $sdir$fn $fn
                        ) | mailx -s "New file attached" "you@me.com"
                        mv $sdir$fn /home/sftp_processed/[$ts]$fn
                        fn=`ls -t $sdir |head -1`
                        fc=`ls $sdir |wc -l`
                done
else
        echo "No files found"
fi

Thank you! :slight_smile:

2

lsof will allow you to see if another process has a given file open.

3

I don't know if it would impose an unacceptable delay into the script, but a sleep might help. Check the filesize, sleep for 30 (or some other reasonable value) seconds, check filesize again, and compare. If the value is changed, rinse and repeat; if not, move the file.

4

This is exactly what I needed. Thank you!

5

lsof seems like a neater solution. I still might still use sleep to delay processing if lsof finds an open file ...or I'll just skip the file altogether and let it be processed at the next interval.

Thanks for the suggestions!