Need of shell script to download data using ftp

Shell Programming and Scripting
1

Hi !

I am just wondering shell script to download data from ftp...

I have text file containing ftp address...looks like this

ftp://site...../filename.xyz
ftp://site...../filename.xyz
ftp://site...../filename.xyz
ftp://site...../filename.xyz

script has to read ftp address and then it has to download data using ftp...after downloading all data files, it should move all data to some directory, which will be having data files of last month, so basically it has to replace old files with new downloaded file..

2

Look at wget

wget --user=vivek --password='myPassword' ftp://site...../filename.xyz
3

You can modify this script to take parameters and download instead of upload...

#!/bin/sh
#THIS SCRIPT CONNECTS TO THE FTP SERVER
PASSWD='xxx'
USER='xxx'
full_file="$1"

cp $full_file . 1> /dev/null 2> /dev/null
ftp -niv ftp.xxx.com<<EOF 1> /dev/null 2> /dev/null
user $USER $PASSWD
binary
passive
cd pub
cd docs
put $1
bye
EOF
4

Hi , I am very new to this site and as a shell script programmer also i am very novice . can you please tell me how its working .

Thanks in advance.

6

If your download data is in url format, then wget is easier, no need to parse host, path and file.

Something like:

# download.txt like
# ftp://somehost/somepath/somefile

mkdir Old 2>/dev/null  # for previous version

while read url
do
     # make filename for url , replace chars / => _
     urlfile=${url//\//_}
     echo "Download $url to file $urlfile"
     cp -f "$urlfile" Old  2>/dev/null    # save old file
     rm -f "$urlfile"  2>/dev/null   # remove previous
     wget -q -O "$urlfile" --user xxxx --password yyyyyy "$url"
     # or anonymous site
     # wget -q -O "$urlfile" "$url"
done < download.txt # input file, include urls
7

Hi kshji... I tried your script not working....please see the attachment...

after downloading all the files I want to replace with last time downloaded files in my directory

/home/nex/ftp_monthly

8

Did you change my solution ? You need.

  • anonymous wget using
  • input file include empty lines - remove or handle those lines in script
  • use bash or ksh93 shell
  • before run the script, cd /home/nex/ftp_monthly
9

I have updated this based on some of the feedback from Corona688

#WRITTEN BY MVONA; FREE TO USE AND ABUSE
#THE WORK "WORKS FOR ME" TM
#THIS SCRIPT CONNECTS TO FTP SERVERS AND DOWNLOADS STUFF
#YOU NEED FOUR COLUMN SPACE SEPARATED FILE CONTAINING IN THE FOLLOWING 
#FORMAT:
#<FTP SERVER ADDRESS> <FULL PATH AND FILE NAME OF THE FILE YOU WANT><LOGIN><PASSWORD>
#
#THIS SCRIPT NEEDS TO BE CALLED BY PASSING THE FULL PATH TO THE ABOVE FILE AS ITS FIRST PARAMETER.
##########################################################
#      ERROR HANDLING                                    #
##########################################################
usage()
    {
    
        echo "Proper usage: $0 <full path to server list>"
        exit 1
    }


if [ $# -ne 1 ]; then
    {
        usage
    }
    elif  [ ! -f $1 ]; then
    {
        echo "I see no file $1"
        usage
    }
    elif [ `awk '{print NF; exit}' $1` -ne 4  ]; then
    {
    echo "I see the file $1, but I don't see 4 columns of data in it..."
        usage
        
    }
fi
##############################################################
#        MAIN ROUTINE                                                         #
##############################################################

cat $1 | grep -v '^#' | (while read server file user pass
do
    echo -e "user $user $pass\n binary\n passive\n get $file\n bye" | ftp -niv $server
done)

Note I added the ability to add comments to the server list by placing a grep statement between cat and the ftp statement, which is why I prefer this method over the simple redirect. I also prefer the echo -e statement since it's easier to code a second case structure later if, for instance, you'll need to use passive and active on other ftp servers.

So now an example server list file might look like this:

#POUND COMMENTS THE LINE
#FORMAT: SERVER FILE USER PASS
ftp.yoursite.com /pub/docs username password
10

That is a useless use of cat and useless use of awk, the read builtin does not need cat's help to read files, does not need awk's help to split tokens, and does not need an entire subshell to do a loop.

The -r is to prevent read from doing things if someone has a backslash in their password.

Also, many systems do not have echo -e, I suggest a here-document instead, which should work anywhere.

while read -r server file user pass
do
        ftp -niv $server <<EOF
user $user $pass
binary
passive
get $file
bye
EOF
done < $1