How archive the older than 30 day files to another unix server

Shell Programming and Scripting
1

I need to archive the older than 30 day file to another uinx server.I have wrote the below uinx script.

for LOOK_DIR in /TempFiles
do
     for FILE in `find ${LOOK_DIR} -mtime -30 -exec ls {} \;`
     do
          echo ${FILE} >> file_list ## This file will have the list of files copied and removed
     done
done
#*********************************************************#
# Copy the files from share drive to Unix server #
#*********************************************************#
ArchFiles_ftp_put
ArchFiles_ftp_put()
{
     /usr/bin/ftp -v -n << EOF > ${FTP_LOG_FILE}
     open ${ARCH_SERVER}
     quote USER ${ARCH_USERID}
     quote PASS ${ARCH_PASSWORD}
     passive
     ascii
     prompt off
     cd ${REMOTE_DIR}
     lcd ${DATA_DIR}
     mput ${FILE}
     bye
     EOF
     EOF
}

When I verify the file list form above script, I am getting file directory along with the file name,
Can you please suggest me the how can I select only older than 30 day files

2

So your problem is to cut off the path from the filename?

Edit:
I just saw you are using -30. Why not try +30? Did you write that script?

3

yes i need to cut off form the path

4 
...
echo ${FILE##*/} >> file_list ## This file will have the list of files copied and removed
...
5

as per you note i have include the +30

echo ${FILE##*/} >> file_list

based on the above code we will get only file name.
but when i am calling ftp function i getting file name along with directory
can you please suggest me where i need to update the script

6

I asked you via PM to use code tags, sent a video with a guide and added a mod comment. You got another PM with a guide.

That's what you asked for.

Define the variable after writing cut off names to the file:

...
echo ${FILE##*/} >> file_list ## This file will have the list of files copied and removed
FILE=${FILE##*/}
...
7

Thanks zaxxon for the qick respose....

I have one more question

When I doing the less than 30 day of file. so I am getting the multiple files.

I need to iterate the loop can you help on this

I have code like this can you please check this.

for ( i=1 ; i<="$FILE� ; i++ ))
     ArchFiles_ftp_put()
     {
          /usr/bin/ftp -v -n << EOF > ${FTP_LOG_FILE}
          open ${ARCH_SERVER}
          quote USER ${ARCH_USERID}
          quote PASS ${ARCH_PASSWORD}
          passive
          ascii
          prompt off
          cd ${ARCH_REMOTE_DIR}
          lcd ${DATA_DIR}
          mput ${FILE}
          bye
          EOF
          EOF
     }
done
8

At first, you should not ignore mod comments or PMs about using code tags. Because when you hit 20 points, your account will be set to "read-only" automatically. In worst cases it can result in a ban.

Iterating through a list of files and then opening a ftp connection for each of them, I would suggest, you just move the files to be transferred into a different directory and just ftp them with mput * all at once which is much more effective, if space on disk permits this.
Or if possible, think about using scp or rsync and move them by this. ftp is not encrypted and does not offer as much options.
Also 2 times EOF is not needed - once is enough. i<="$FILE� does not make sense to me as you are comparing a number with a string.

9

you may use

basename $FILE # This will give you the filename onle

 
for LOOK_DIR in /TempFiles
do
for FILE in `find ${LOOK_DIR} -mtime -30 -exec ls {} \;`
do
echo ${FILE} >> file_list ## This file will have the list of files copied and removed
ArchFiles_ftp_put ${FILE}
done
done
#*********************************************************#
# Copy the files from share drive to Unix server #
#*********************************************************#
#ArchFiles_ftp_put
ArchFiles_ftp_put()
{
/usr/bin/ftp -v -n << EOF > ${FTP_LOG_FILE}
open ${ARCH_SERVER}
quote USER ${ARCH_USERID}
quote PASS ${ARCH_PASSWORD}
passive
ascii
prompt off
cd ${REMOTE_DIR}
lcd ${DATA_DIR}
mput $1
bye
EOF
EOF
}
10

as per the above code i am getting error as can not open the file

archive_file.ksh[9]: : cannot open
+ echo test_sample.txt
+ 1>> file_list
+ ArchFiles_ftp_put test_sample.txt

---------- Post updated at 10:14 AM ---------- Previous update was at 09:58 AM ----------

as per your suggesion next time on wards i will use code tags.

11

Hi ,
Have made some changes in the code and below is working fine for me.

#!/bin/ksh
ARCH_SERVER=xxxxxx
ARCH_USERID=xxxxxxx
ARCH_PASSWORD="xxxxxxx"
REMOTE_DIR=xxxxxxx
FTP_LOG_FILE=xxxxxxx/TEST/FTPLOGa
echo  > $FTP_LOG_FILE

#*********************************************************#
# Copy the files from share drive to Unix server #
#*********************************************************#
ArchFiles_ftp_put()
{
/usr/bin/ftp -v -n << EOF >> ${FTP_LOG_FILE}
open ${ARCH_SERVER}
quote USER ${ARCH_USERID}
quote PASS ${ARCH_PASSWORD}
passive
ascii
prompt off
cd ${REMOTE_DIR}
lcd ${DATA_DIR}
mput ${FILE_NAME}
bye
EOF
}
for LOOK_DIR in /cb/home/n421522/TEST
do
for FILE in `find ${LOOK_DIR} -mtime -30 -exec ls {} \;`
do
echo ${FILE} >> file_list ## This file will have the list of files copied and removed
DATA_DIR=`dirname ${FILE}`
FILE_NAME=`basename ${FILE}`
ArchFiles_ftp_put ${FILE}
done
done

You can also add date to the FTP_LOG_FILE generated to retrive it for longer time.

Please let me know if you get struck somewhere.

Rgds,
Aashish

12

Thanks Aashish for the replay

I wrote the scipt as below:
#!/bin/ksh
set -x
ArchFiles_ftp_put()
{
        /usr/bin/ftp -v -n << EOF > ${FTP_LOG_FILE}
        open ${ARCH_SERVER}
        quote USER ${ARCH_USERID}
        quote PASS ${ARCH_PASSWORD}
         passive
        ascii
        prompt off
        cd ${ARCH_REMOTE_DIR}
        lcd ${DATA_DIR}
        mput ${FILE_NAME}
        bye
 EOF
EOF
}
#**************************************************#
# Main                                             #
#**************************************************#
if [ $# -ne 5 ]
then
echo "Invalid number of parameters::"
echo "Please pass 5 parameters in following order:Host Name, UserID, Password, RemotDir and project dir::"
exit 1
fi
#***************************************************************#
# Set "FTP" variables                                           #
#***************************************************************#
ARCH_SERVER=$1
ARCH_USERID=$2
ARCH_PASSWORD=$3
ARCH_REMOTE_DIR=$4
PROJ_NAME=$5
#**************************************************#
# Set directory values                             #
#**************************************************#

for LOOK_DIR in /tmp/arch_files/
do
for FILE in `find ${LOOK_DIR} -mtime -30 -exec ls {} \;`
do
echo ${FILE}>> file_list ## This file will have the list of files copied and removed
DATA_DIR=`dirname ${FILE}`
FILE_NAME=`basename ${FILE}`
ArchFiles_ftp_put ${FILE}
done
done
FTP_LOG_FILE="/tmp/Log/FTP_put_Archive_Files.log"
echo "Starting...">$FTP_LOG_FILE

When i ran the above scipt i am geeting below error message, can you please check the script.

"
archive_file.ksh[5]: : cannot open
+ echo test_sample.txt
+ 1>> file_list
+ dirname test_sample.txt
+ DATA_DIR=.
+ basename test_sample.txt
+ FILE_NAME=test_sample.txt
+ ArchFiles_ftp_put test_sample.txt
+ /usr/bin/ftp -v -n
+ 0<< \EOF
        open usmanasdsbul01
        quote USER testuser
        quote PASS testuser
         passive
        ascii
        prompt off
        cd /proj_name/Archive
        lcd .
        mput test_sample.txt
        bye
        EOF
EOF
archive_file.ksh[5]: : cannot open
+ echo tmp.txt
+ 1>> file_list
+ dirname tmp.txt
+ DATA_DIR=.
+ basename tmp.txt
+ FILE_NAME=tmp.txt
+ ArchFiles_ftp_put tmp.txt
+ /usr/bin/ftp -v -n
+ 0<< \EOF
        open usmanasdsbul01
        quote USER testuser
        quote PASS testuser
         passive
        ascii
        prompt off
        cd /proj_name/Archive
        lcd .
        mput tmp.txt
        bye
        EOF
EOF
"

---------- Post updated at 07:21 AM ---------- Previous update was at 05:05 AM ----------

Thanks zaxxon and aashish
i have resolved the problem.. as per the zaxxon suggestion I moved the older than the 30 day file to another folder and I called mput option it's working fine.
Thanks you so much for your suggestion s and code.

13

Thanks for update...Rather than using ftp for each file why don't you zip all the files together and ftp the compressed file only.
This will be much more effective !