I have a weird requirement where I have to get the files from a FTP(Lets say FTP1) location and place it on my current FTP(Lets say FTP2) location. The issue is, these are daily files (in a pattern Sales_YYYYMMDD_report.csv) and are placed every day on FTP1 and my process usually runs on Monday(eg. 09-Sept-2013) which has to use the file of the previous week starting from Sunday(eg. 01-Sept-2013) to Saturday(eg. 07-Sept-2013) place them on FTP2 location and then run the Informatica process. For an instance, if I run the process on Monday,09-Sept-2013, I have to pull all the files from FTP1 which have file names such as
Sunday file --> Sales_20130901_report.csv
Monday file --> Sales_20130902_report.csv
Tuesday file --> Sales_20130903_report.csv
Wednesday file --> Sales_20130904_report.csv
Thursday file --> Sales_20130905_report.csv
Friday file --> Sales_20130906_report.csv
Saturday file --> Sales_20130907_report.csv
How can I achieve this in a shell script? I know the part to get the files from another FTP, but I am not sure how to get the 7 files.
P.S: I cannot use the file creation/last modified timestamps to get the files. Irrespective of the created timestamp and the day I run my Informatica process, I have to get the files which have last week's dates in the file names and put in my FTP2 location and then continue with them.
Please help...
This is an example script script that creates a 'ftp' script that 'puts' the files from the last sunday thru saturday to remote host:
echo 'ftp -vn FTP2 <<ENDSCRIPT' >>ftp_script.sh
echo 'user userid pass' >>ftp_script.sh
echo 'cd /path/to/dir' >>ftp_script.sh
echo 'lcd /path/to/dir' >>ftp_script.sh
echo put Sales_`date -d '-8day' +%Y%m%d`_report.csv >>ftp_script.sh
echo put Sales_`date -d '-7day' +%Y%m%d`_report.csv >>ftp_script.sh
echo put Sales_`date -d '-6day' +%Y%m%d`_report.csv >>ftp_script.sh
echo put Sales_`date -d '-5day' +%Y%m%d`_report.csv >>ftp_script.sh
echo put Sales_`date -d '-4day' +%Y%m%d`_report.csv >>ftp_script.sh
echo put Sales_`date -d '-3day' +%Y%m%d`_report.csv >>ftp_script.sh
echo put Sales_`date -d '-2day' +%Y%m%d`_report.csv >>ftp_script.sh
echo quit >>ftp_script.sh
echo 'ENDSCRIPT' >>ftp_script.sh
echo 'rc=$?' >>ftp_script.sh
echo 'if [[ $rc != 0 ]]; then' >>ftp_script.sh
echo ' echo "Error occured...$rc" `date "+%Y-%m-%d-%H.%M.%S"`' >>ftp_script.sh
echo 'else' >>ftp_script.sh
echo ' echo "Successful transfer...$rc" `date "+%Y-%m-%d-%H.%M.%S"`' >>ftp_script.sh
echo 'fi' >>ftp_script.sh
$ cat ftp_script.sh
ftp -vn FTP2 <<ENDSCRIPT
user userid pass
cd /path/to/dir
lcd /path/to/dir
put Sales_20130901_report.csv
put Sales_20130902_report.csv
put Sales_20130903_report.csv
put Sales_20130904_report.csv
put Sales_20130905_report.csv
put Sales_20130906_report.csv
put Sales_20130907_report.csv
quit
ENDSCRIPT
rc=$?
if [[ $rc != 0 ]]; then
echo "Error occured...$rc" `date "+%Y-%m-%d-%H.%M.%S"`
else
echo "Successful transfer...$rc" `date "+%Y-%m-%d-%H.%M.%S"`
fi
Hi Spacebar,
Your code is very helpful. However, I see in your code that, you are subtracting the dates as if running the process on Mondays only, but there are few scenarios where I run the job on Tuesdays or any day of the week too (eg. due to long weekends or holidays or sometimes the files are not ready on the FTP1). Can you please suggest a different code(or modification to the current one) to get the files for the previous week no matter which day or when I run the job in the current week.
Thanks
---------- Post updated at 02:04 PM ---------- Previous update was at 11:26 AM ----------
Hi Spacebar,
I got the answer to my question :). I used the following:
date -d "last saturday -1day" +%Y%m%d
Thanks
Dhruuv.