To merge files on FTP site before getting them

Shell Programming and Scripting
1

I have a job that connects to FTP site, rename, get, delete a file and then process it. The renaming is done in consideration of this is one file name into which data is being accumulated many times within 24 hour cycle.

This is what happens now:

 ftp -i $FTPHOST <<-EOF
        ren INPFILE $OUTF
        get $OUTF
        del $OUTF
EOF

We got another accumulator file that needs to be brought over and processed together with the original one. We can bring it in in a similar fashion, then merge locally.

My question is there an alternative, to connect to the site, rename both and somehow merge them there and bring in as one file ready to be processed?

2

Not quite sure why you are renaming a remote file only to delete it afterwards:

 ftp -i $FTPHOST <<-EOF
        get INPFILE $OUTF
        del INPFILE
EOF

I'm not sure you can merge two files remotely with FTP (others can correct me on this) but if you were to use lftp instead, which is much better for scripting FTP as it allows for recovery after failures, you could investigate its cat command:

cat INPFILE1 INPFILE2 > $OUTF

which when used interactively will save the files locally. I have not tested within a script, however.

Andrew

3

Thanks for lftp suggestion, but I don't have control over remote systems.

The rename - get - delete scheme works well when your file is being frequently updated on the remote system and you don't want to get a file with some data half-written in.

4

lftp is a client program that you would run on the local system.

Okay, that makes sense.

Andrew