Picking up last two specific files, for transfer

Hi

I need to transfer files with csv extension from one server to another (all running solaris ).
But the files to be picked are in a directory that also has:

/directory
deposit_698.csv
deposit_698.checksum
deposit_699.csv
deposit_699.checksum
deposit_700.csv
deposit_700.checksum
Withdrawal_697.csv
Withdrawal_697.checksum
Withdrawal_698.csv
Withdrawal_698.checksum
Withdrawal_699.csv
Withdrawal_699.checksum
Withdrawal_700.csv
Withdrawal_700.checksum

In this case I need to transfer the deposit_700.csv and the Withdrawal_700.csv to a deposits folder on another server and withdrawls folder.

My script is as follows:

#!/bin/sh
HOST=192.168.6.25
USER=rico
PASSWD=123xyz
/usr/bin/ftp -inv <<EOF
open $HOST
user $USER $PASSWD
binary
cd /home/scripts/Live deposits/input
mput *Deposit_* | head -n 1*
cd /home/script/Live withdrawal/input
mput *Withdrawal_* | head -n 1*
bye
EOF

My first problem is that one of the directory name in the destination server has a space on the directory name Live deposits and live withdrawals .
the second problem is that I am not able to transfer the last Deposit_700.csv and Withdrawal_700.csv .

my files are going to one folder before the folder with space on its name

---------- Post updated at 12:31 PM ---------- Previous update was at 12:04 PM ----------

Hi

I solve my first problem by using:

cd /home/scripts/Live\ deposits/input

Is that always the same running number, i.e. 700 in above? Are the files created in that sequence, i.e. *700* are the newest? Do respective deposit_* and Withdrawal_* files have identical timestamps?

No, everyday a new files are generated, so tomorrow will be Deposit_701.csv and Withdrawal_701 , so every single day, there will be a new file

You could remove the OPEN command by putting the hostname on the ftp line:-

/usr/bin/ftp -inv $ HOST <<EOF

The files you have are local (your client sends) so work them out before starting the FTP.

Would you not be better with tail?

#!/bin/sh

# Determine files first
D_file=$(ls -1 *Deposit_*" | tail -1)
W_file=$(ls -1 *Withdrawal_*" | tail -1)

# Call the FTP
HOST=192.168.6.25
USER=rico
PASSWD=123xyz
/usr/bin/ftp -inv $HOST <<EOF
user $USER $PASSWD
binary
cd /home/scripts/Live deposits/input
put "$D_file"
cd /home/script/Live withdrawal/input
put "$W_file"
bye
EOF

Does that help? If it doesn't work, please post your output/errors.

Kind regards,
Robin

I did a test, outside the script of the following line:

 ls -1 *Withdrawal_*" | tail -1
>

what come out was >

loose the "

after your suggested changes, on the other server (destination) the output is strange:

ls -lrt
total 2
-rw-r--r--   1 rico  rico       32 Jun 26 15:33 |

Please show us your complete script as it currently stands.

It looks like you have confused ftp into thinking that a vertical bar character ( | ) should be used as a filename instead of as a separator in a shell pipeline.

Whoooops! I think there is a mistake in my suggestion. The expression to match the filenames should be wrapped in double quotes, " and somehow I've lost the beginning out. Try this amended version with the addition in red:-

#!/bin/sh

# Determine files first
D_file=$(ls -1 "*Deposit_*"    | tail -1)
W_file=$(ls -1 "*Withdrawal_*" | tail -1)

# Call the FTP
HOST=192.168.6.25
USER=rico
PASSWD=123xyz
/usr/bin/ftp -inv $HOST <<EOF
user $USER $PASSWD
binary
cd /home/scripts/Live deposits/input
put "$D_file"
cd /home/script/Live withdrawal/input
put "$W_file"
bye
EOF

Apologies,
Robin

Robin,
I think you need to loose the double-quote to allow shell to do name expansion.

Hi Robbin,
vgersh99 is correct in noting that quoted asterisks, question marks, etc. are matched by the shell as literal strings. To use asterisks to match any string of characters in a pathname and to use a question mark to match any single character in a pathname in pathname pattern expansions, the asterisks and question marks cannot be quoted.

Since there are no IFS characters in either Deposit_ or Withdrawal_ , no quotes are required. But, any of the following would work:

D_file=$(ls -1 *"Deposit_"*    | tail -1)
W_file=$(ls -1 *"Withdrawal_"* | tail -1)

D_file=$(ls -1 *'Deposit_'*    | tail -1)
W_file=$(ls -1 *'Withdrawal_'* | tail -1)

D_file=$(ls -1 *Deposit_*    | tail -1)
W_file=$(ls -1 *Withdrawal_* | tail -1)

unless the pathnames roll over from Deposit_999 to Deposit_1000 and Withdrawal_999 to Withdrawal_1000 .

Hi fretagi,
If you are trying to find the last two files based on most recent modification timestamp instead of by alphanumeric filename sort order consider trying one of the following instead:

D_file=$(ls -1rt *Deposit_*    | tail -1)
W_file=$(ls -1rt *Withdrawal_* | tail -1)

D_file=$(ls -1t *Deposit_*    | head -1)
W_file=$(ls -1t *Withdrawal_* | head -1)

Either of these will work even when the number of digits increases in the matched filenames.

Hi Don!

I tried the code outside the script just like this:

ls -1rt *Deposit_*    | tail -1
Deposit_699.checksum

and

ls -1rt *Withdrawal_* | tail -1
Withdrawal_700.checksum

It should not pick the .checksum extension files only the .csv files.

---------- Post updated at 07:32 AM ---------- Previous update was at 07:13 AM ----------

Hi

my original script was:

#!/bin/sh
HOST=192.168.6.25
USER=rico
PASSWD=123xyz
/usr/bin/ftp -inv <<EOF
open $HOST
user $USER $PASSWD
binary
cd /home/scripts/Live deposits/input
put *Deposit_* | tail -1
cd /home/script/Live withdrawal/input
put *Withdrawal_* | tail -1
bye
EOF

which gave the following output

-rw-r--r--   1 fundamo  fundamo     3723 Jan  1 00:00 Withdrawal_625.csv

which is wrong because the file I want is last Withdrawal_700.csv

Now using the code suggested by a forum member:

#!/bin/sh

# Determine files first
D_file=$(ls -1 *Deposit_* | tail -1)
W_file=$(ls -1 *Withdrawal_* | tail -1)

# Call the FTP
HOST=192.168.6.25
USER=rico
PASSWD=123xyz
/usr/bin/ftp -inv $HOST <<EOF
user $USER $PASSWD
binary
cd /home/scripts/Live deposits/input
put "$D_file"
cd /home/script/Live withdrawal/input
put "$W_file"
bye
EOF

It gives me the following results:

./teste.sh: syntax error at line 4: `D_file=$' unexpected

So filter for csv files:

ls -1t *Deposit_*.csv | head -1
ls -1t *Withdrawal_*.csv | head -1

and personally I'm in favour of using head over tail (imagine your directory has 300 files in it ...)

Your version of /bin/sh does not recognise the more modern process substitution syntax:

D_file=$(ls -1 *Deposit_* | tail -1)

You could either modify the "shebang" line at the top of your script to use ksh if you have it, or bash , or alternatively use the older process substitution syntax using backticks:

D_file=`ls -1 *Deposit_* | tail -1`

Andrew

Replacing D_file=$(ls -1 *Deposit_* | tail -1) with D_file=`ls -1 *Deposit_* | tail -1` worked fine

---------- Post updated at 11:59 AM ---------- Previous update was at 11:32 AM ----------

there is a need to also include the Deposit_699.checksum and Withdrawal_700.checksum . I did include in the script like this:

D_file=`ls -1 *Deposit_* | tail -1`
Dd_file=`ls -1 *Deposit_*.checksum | tail -1`
W_file=`ls -1 *Withdrawal_* | tail -1`
Ww_file=`ls -1 *Withdrawal_*.checksum | tail -1`

but this did not work out..

I know that you don't believe me yet, but about 299 cycles from now (when your numbers grow to 4 digits instead of 3 digits) you will find that your code stops working unless you pay attention to my suggestion...

D_file=`ls -1rt *Deposit_*.csv | tail -1`
Dd_file=`ls -1rt *Deposit_*.checksum | tail -1`
W_file=`ls -1rt *Withdrawal_*.csv | tail -1`
Ww_file=`ls -1rt *Withdrawal_*.checksum | tail -1`

I know you say you prefer tail over head , but your script will run faster if you use:

D_file=`ls -1t *Deposit_*.csv | head -1`
Dd_file=`ls -1t *Deposit_*.checksum | head -1`
W_file=`ls -1t *Withdrawal_*.csv | head -1`
Ww_file=`ls -1t *Withdrawal_*.checksum | head -1`

and on Solaris systems, you should get the same results without the -1 option (filtered either by head or by tail ):

D_file=`ls -t *Deposit_*.csv | head -1`
Dd_file=`ls -t *Deposit_*.checksum | head -1`
W_file=`ls -t *Withdrawal_*.csv | head -1`
Ww_file=`ls -t *Withdrawal_*.checksum | head -1`

Note that the ls -1 option is on by default whenever standard output is not a terminal (and the pipe that is the standard output of ls when its output if fed to head or tail is not a terminal).

I have followed your suggestion, (using the head ) and its working fine, thank you.