FTP'ing the zipped file

Shell Programming and Scripting
1

Hi,

I need to have a shell script that FTP's a zipped file from a particular location.

I have some path and inside that path i will have folders like x_timestamp and inside x_timestamp there may many folders based upon events like y_111,y_222,y_333.Inside each event there will be another folder z.

Inside the z only the file to be FTP'd is lying.

And x_timestamp folder will not be deleted as soon as we take the zipped file. So as soon as the file x_20110509 (Say for eg)is FTP'd the next time eventhough the folder x_20110509 is there the script should go for other x_timestamp folder leaving the one which is already FTP'd.

Thanks in advance.

2

So do you want to pick up newer files of name x_timestamp/y111/z/*.zip? If so, we might be able to achieve it with a two call ftp by using a logging file. Could we consider the following logic:-

  1. List out all the files that fit the criteria, e.g. */*/*/*.zip
  2. Exclude all the files that we already have
  3. Get the remainder
  4. Update the logging file

Would that suit? If so, then perhaps this will help:-

#!/bin/ksh
#
# Get new files only ftp
# 

source=source.mycompany.com
basesource="/my_source"
search="*/*/*/*.zip"
register=received_files.txt
tempfile=dir_listing.txt
gets=get_commands.txt
username=fred
password=fred

ftp -n $source > $tempfile || -EOFTP
 user $username
 $password
 cd $basedir
 dir $search
 quit
EOFTP

# Okay, we now have a list of all interesting files on the remote source
# We might need to trim this down with head/tail commands depending on the FTP
# server response to just leave the file records.

grep -vf $register $tempfile|while read line
do
   file="${line##* }"      # Get last field, i.e. file name.  Format is hash, hash, asterix, space
   echo get $file
done > $gets

# Now we have the required gets for new or updated files so do the actual transfer

ftp -n $source || -EOFTP
 user $username
 $password
 cd $basedir
 bin                  # If you want a binary transfer for compressed files
 `cat $gets`
 quit
EOSQL

# Update the registered files to show that they have transferred
grep -vf $register $tempfile >> $register

Caveat/warning
I must admit that this has not been tested as I'm not sure of your requirements. There is also no error checking in the ftp transfer, or indeed the script as a whole. There may be some syntax errors, but hopefully this free suggestion may get you started down the right path.

Like I comment in the code, some adjustment may well be needed to exclude directory listing headers/trailers, or perhaps you could just accept errors for the first run then they will get ignored. I hope that this helps, but let us know how you get on and someone else may provide a better suggestion or correct my errors.

Robin
Liverpool/Blackburn
UK

3

Thanks Robin for the code

5

Hi Robin,

The command dir */*/*/*.zip is not working after logging into FTP

Do we have anyother command that can dynamically get the file names in the above mentioned layers.

Thnx,
Vinoth