I have to send a processed file to 24 different server locations. I feel, if this job can be done parallel in the background - Time will come down.
I found the script to FTP the file for a single server location through past Unix posts as below:
#!/usr/bin/ksh
ftp -v -n "ipaddress"
user "Username" "Username"
cd /distant/directory
put filename
If error >> ${LOG}
mail -x john@gmail.com -s FTP failed to the server ${ipaddress}
else
echo "FTP Successful to the server ${ipaddress} >> ${LOG}
quit
How to automate the script to send the Source File to Different FTP Locations.
For eg: If I create a file with all the 24 different FTP Target Locations in "Filelist.txt" as below:
Thank you very much. I will try it tomorrow at office.
As per my understanding, this script reads a file: filelist.txt
and pass the 3 parameters(24 lines, 24 times) to the calling function: ftpscript right.
Is it possible to do this FTP parallely to reduce time.
You can add an ampersand (&) to the end of the ftpscript line, and that will release each copy to run independently. Depending upon where these files are going on your network, and the network speed, there may or may not be much difference in total throughput, and having each connection running more slowly might cause timeout errors.
One thing that I noticed, you have commas separating the fields in the input data file. Either use a text editor to replace these commas with spaces, or add
i tried the below script as you suggested. But i am not able to work it out.
When i run the script, It stops at the Message: "FTPing the file from the server". not going further. Please tell me what i am doing wrong.
etluser : 24ftplocations_i.sh
FTPing the file from the server
Script(24ftplocations_i.sh) code is as below:
#!/usr/bin/ksh
export filename=/grid/PowerCenter/stage/velocity_r3_dev/inbound/ATRPU/filelist.txt
export SOURCE_DIR=/grid/PowerCenter/stage/velocity_r3_dev/inbound/ATRPU
export ICOMS_FTP_TGT_DIR1=/dw/cdrsp/stage/velocity/vmedia/input/ATRPU/ICOM_SERV1
export ICOMS_FTP_TGT_DIR2=/dw/cdrsp/stage/velocity/vmedia/input/ATRPU/ICOM_SERV2
export FILE_MASK="ATRPU_RP_ATU"
ftp_data_file()
{
# local parameters
local FTP_FUNCTION=$1
local FILE_NAME=$2
local FTP_SERVER=$3 # remote delivery server
local FTP_USER=$4 # ftp user name
local FTP_PASS=$5 # ftp user pwd
local FTP_TARGET_DIR=$6 # local target file directory
# local variables
local SOURCE_DIR=${SOURCE_DIR}
echo "FTPing the file from the server"
# ftp file from remote server to local informatica server
ftp -nv ${FTP_SERVER} <<-!!
user ${FTP_USER} ${FTP_PASS}
cd ${FTP_TARGET_DIR}
ascii
prompt
${FTP_FUNCTION} ${FILE_NAME}
bye
!!
co_error_check $? "Failed to ${FTP_FUNCTION} file ${SOURCE_DIR}/${FILE_NAME} to ${FTP_TARGET_DIR}"
}
# main function
while IFS="," read ipaddress username password
do
# ftp file from remote server to local server
cd ${SOURCE_DIR}
ftp_data_file mput ${FILE_MASK}*.csv ${ipaddress} ${username} ${password} ${ICOMS_FTP_TGT_DIR1}
ftp_data_file mput ${FILE_MASK}*.csv ${ipaddress} ${username} ${password} ${ICOMS_FTP_TGT_DIR2}
done <$filename
The << characters in the ftp statement indicate that the input for the ftp command will come from the lines following this line until an end of input marker is found.
The end of input marker has to start in column one.
Also add the -i option to the ftp command to suppress confirmation on mput
I believe Jit is very Busy and he helped me to come till this point.Can any one help me on this as it is urgent.
Thanks,
meruga
---------- Post updated at 08:50 AM ---------- Previous update was at 08:48 AM ----------
Sorry Jit,
I haven't seen your reply. It is too tight situation from end. i will work on it.
Thanks
---------- Post updated at 09:05 AM ---------- Previous update was at 08:50 AM ----------
Hi Jgt,
Removed the header row(ipaddress,username,password) from the "filelist.txt" file
I have changed my script lines as below:
ftp -nv -i<<-!!
${FTP_SERVER}
user ${FTP_USER} ${FTP_PASS}
cd ${FTP_TARGET_DIR}
ascii
prompt
${FTP_FUNCTION} ${FILE_NAME}
bye
!!
And ran the script as below:
etl-kn-t2-etluser : 24ftplocations_i.sh
start
main before while
ip address is 172.22.234.55
username is cipxcd
password is spffg
- FTPing the file from the server
?Invalid command
Not connected.
Not connected.
Not connected.
Interactive mode on.
Not connected.
24ftplocations_i.sh[28]: co_error_check: not found
- FTPing the file from the server
The .netrc file exists in the user's home directory and should have permissions of 0600.
The content of the file is:
No DNS resolution is done, so the host name on the ftp command line must match the machine name in the .netrc file.
Simplify your script; instead of trying to write the universal solution, put literals into it until you get it to work, get rid of the function, and write an inline solution.
Use building blocks. Get the ftp portion itself to work without any looping, and any variable substitution. Add variables one variable at a time.
Try running the ftp portion interactively, without the re-direction (leave the <<EOF) off the line, and without the -n option.
You will be prompted for the userid and password.
Since you are only sending one file, you can use "put" instead of "mput"
If both systems use the same operating system, you can omit the 'ascii' directive, it is only used to covert line endings in text files from CRLF to LF and vice versa.