Ftp with bash, append file where left off

Shell Programming and Scripting
1

I'm working on a bash script to finish uploading a file.

I need a way to get $filesize so that "restart $filesize" will work.

Here is my script:

ftp -n -v <<END_SCRIPT
open ftp.$domain
user $user@$domain $password
size $file
restart $filesize
put $file
quit
END_SCRIPT

Wayne Sallee
Wayne@WayneSallee.com

2

Try ls -l filename which may work in some ftp servers. You may need to do that in a different ftp <<END_SCRIPT block so you can save its value to process it into something you can use.

3

Thanks!

Yes it accepts that command, It won't create the file. So anyone else wanting to do this, remember to create the file for it to save to first.

output to local-file: filename?

Press enter :slight_smile:

I wish the edit post function had a preview button. Now the preview button is showing up. Sometimes it shows up, and sometimes not. :slight_smile:

Wayne Sallee
Wayne@WayneSallee.com

---------- Post updated at 04:31 PM ---------- Previous update was at 03:54 PM ----------

It looks like I'll be able to make this work.

Thanks!

I'll post back later with some sample code for anyone else searching the net with the same need.

Wayne Sallee
Wayne@WayneSallee.com

4

Check if your FTP client has a batch mode, that may avoid it prompting for local file names. It might be -b, not all ftp clients are the same.

5

Hi.

Utility ncftp allows option -z to resume uploads and downloads.

I usually use sftp , so I have not used ncftp for years, but you may find it to be easier than bare-bones ftp for tasks like this... cheers, drl

6

Another choice might be rsync which I believe can be used to move the differences rather than the whole file. I've not used it though, so I can't be sure. Could anyone else voice an opinion on this?

I might be getting confused with robocopy that I've seen the Windows team setting up. I'm not sure if there is a unix version.

Robin

7

I don't think rsync searches for differences smaller than an entire file, just compares checksums and replaces.

8

I used the prompt command to stop it from prompting.

Thanks everyone for your help.
It's solved.

I'll get some sample code up soon for others searching with this same need.

Wayne Sallee
Wayne@WayneSallee.com

1 Like
9

Hi, Corona688.

I use rsync as part of rsnaphot to do backups 6 times daily. It is very efficient:

Best wishes ... cheers, drl

1 Like
10

Here is the working code simplified:

ftp -n <<EOF
open ftp.$server
user $user@$server $password
prompt
ls -l sizeonserver.txt
EOF
myappendsize=$(awk '/security.mpg/ {print $5;}' sizeonserver.txt)

ftp -n <<EOF
open ftp.$server
user $user@$server $password
restart $myappendsize
put $myfile
EOF

The code that I was working on is a bash file that uploads a file while it is being created.

So for anyone looking for some code to upload to ftp a file while it is being created, here is the code.

#!/bin/sh

thisdir=`dirname $0`
cd $thisdir

user=youruser
server=yoursever.com
password=yourpassword
myfile=yourfile.extention
progcreatingfi=yourprogramcreatingthefile

touch sizeonserver.txt
echo "Ready to upload..."
echo "Buffering...."
sleep 30
echo "Uploading..."
ftp -n <<EOF
open ftp.$server
user $user@$server $password
put $myfile
prompt
ls -l sizeonserver.txt
EOF
myappendsize=$(awk '/security.mpg/ {print $5;}' sizeonserver.txt)

echo "Done first part"
echo "Buffering...."
sleep 30
while pgrep "$progcreatingfi" >/dev/null; do
echo "Appending"
ftp -n <<EOF
open ftp.$server
user $user@$server $password
restart $myappendsize
put $myfile
prompt
ls -l sizeonserver.txt
EOF
myappendsize=$(awk '/security.mpg/ {print $5;}' sizeonserver.txt)

echo "Buffering...."
sleep 30
done

echo "Finalizing upload"
ftp -n <<EOF
open ftp.$server
user $user@$server $password
restart $myappendsize
put $myfile
EOF
echo done
sleep 3

I noticed that there is also a throttle command for ftp, so that might also be useful.

Wayne Sallee
Wayne@WayneSallee.com