Partial file pulling

Shell Programming and Scripting
1

I am connecting to another server through sftp. I am running one batch script to pull file from another server. sometimes i am receiving partial files. I am using below commands in batch script.

ls -ltr new.txt
mget new.txt
bye

The file is of 1 MB only.In most of the cases , i received partial file of 200kB only. Can anyone help me on this issue.

2

What help do you need? I'm afraid we can't help with shaky lines or wires.
You could use the exit code of sftp or its

option to check if the transfer succeeded.

3

Are you sure the file is closed (fully written) when you download it? If the remote system is writing files constantly and you grab one that is only partially written, you would see this problem. Then when you check back on the system, the file write has completed and you see that you got less than the full file because you started copying too early.

RudiC covered the other obvious possible problem.

4

Yes the file is closed and fully written.

I am using below command to bring the file
sftp -b bringFile {Servername}

bringFile contains below commands

cd {path}
ls -l new.txt
ls -l new_1.txt
mget new.txt
mget new_1.txt
bye

I am getting below log message. It is not executing the second mget command at all.

sftp> cd /path/
sftp> ls -l new.txt
-rw-------    0 0        0          745335 Nov 13 00:57 new.txt
sftp> ls -l new_1.txt
-rw-------    0 0        0              28 Nov 13 00:30 new_1.txt
sftp> mget new.txt
Fetching c:/path/new.txt to new.txt

At the end i am checking the size of both files
Size of the pulled file 131072 bytes
Size of the Source File 745335 bytes
The Pulled file size and the Source File size are different.

5

Your script is obviously incomplete since you don't show it saving a logfile anywhere but you say a logfile is saved. Are you trapping sftp's stderr or all stderr?

Your script should check the return code of sftp. It can probably tell if the download was interrupted.

You should also check if something else is killing your script. sftp would not catch that.

trap "echo 'Script was killed for unknown reason' >&2" EXIT
# Trap all errors, from script and sftp, into 'sftplog' file
exec 2>sftplog

TRY=0
MAXTRIES=3

while ! sftp -b bringFile {Servername}
then
        let TRY=TRY+1
        if [[ "$TRY" -gt "$MAXTRIES" ]]
        then
                echo "SFTP download failed, giving up after $MAXTRIES tries" >&2
                exit 1
        fi

        echo "SFTP download had a problem, try $TRY/$MAXTRIES" >&2
fi
6

Thanks for your help.

I am getting the exit code as 255. What might be the reason??

Exit code 0 - successful
Exit code 137 - Session killed

7

man ssh :

What about the - several times - requested log file?