Hi,
Iam using this part of script in a shell script...
#! /usr/bin/ksh
HOST=remote.host.name
USER=whoever
PASSWD=whatever
exec 4>&1
ftp -nv >&4 2>&4 |&
print -p open $HOST
print -p user $USER $PASSWD
print -p cd directory
print -p binary
print -p put tar.gz
print -p bye
I want the output to be on screen and to a logfile.
The Tee wont work bcos Iam not just doing ftp.
Any Ideas...
"Iam using this part of script in a shell script...
The Tee wont work bcos Iam not just doing ftp."
Well you lost me there... 
"I want the output to be on screen and to a logfile."
I understand that. It's gonna be a bit rough...
#! /usr/bin/ksh
#
# get password
print -n password -
stty -echo
read PASSWD
stty echo
echo
HOST=someftphost
USER=joeuser
#
# arrange for all stdout and stderr to be sent to a tee process
exec 5>&1
tee /dev/tty > ftptest.log |&
pid1=$!
exec >&p 2>&1 4>&1
#
# invoke ftp co-process
ftp -nv >&4 2>&4 |&
pid2=$!
print -p open $HOST
print -p user $USER $PASSWD
print -p get foo
print -p bye
wait $pid2
#
# disconnect stdout and stderr from tee process
# send a line to stdout which comes directly from this script
exec >&- 2>&- 4>&- >&5 2>&1
echo
exit 0
Hello Perderabo,
I'd like to ask for further assitance in this regards, If you dont mind.
I have a similar requirement where in I have :-
a) Predefined the desitnation server / hardcoded value.
b) Ask for the TAR file to transfer to the destination server.
c) After the file has been transfered it has to be extracted.
How can i do this and the important question is, how will the extraction and other validations take place via FTP ?
Kindly assist.
Thanks
Perderabo Thanksa lot for the script example , in the script
HOST=damien1
USER=con
PASSWD=
is there a way to mask this password . The reason being that i do the transfer as myself which in this case I have to expose my password, is there a way to encrypt this.
Thanks for your help guys.
These examples are great and sort of do what I want.
Since I'm not a great scriptor can one of you please help me out.
This is what I have.
I have been told I must use ftp and not rsync as rsync is not allowed through the firewalls but ftp is.
Its OK..
I've worked out how to do it...
Using the info gathered from this thread, this is what I have so far..
#! /usr/bin/ksh
#
# Goran Cvetanoski - 07/11/2006
#
# /export/ftpdir/autoftp
#
# This script uses ftp to move files from one server to another.
# It copies files from one servers Outgoing to another servers Incoming
# It FTPs to Incoming/.receiving directory and then moves the files to Incoming
# Once the files have been sent it moves the files from Outgoing to Outgoing/.sent
# It deletes files older then 14 days from both servers Outgoing/.sent directories
# It traps any FTP errors and send an email with the error message
# It can also send emails on successfull transfers, if wanted
#
# Available flags:
# -------------------------------------------------------------------------
# mget - ftp mget from $RHOST:\$ROUT to $LIN
# mput - ftp mput from $LOUT to $RHOST:\$RIN
# xfer - Perform a mget then an mput on $RHOST
#
# Two log files are created:
# -------------------------------------------------------------------------
# ftp.mail - This gets emailed to $ADMIN , if comment removed
# ftp.log - Verbose output from the ftp
#
# -------------------------------------------------------------------------
# A small note on all the fd stuff..
# -------------------------------------------------------------------------
# exec 4>&1
# ftp -nv >&4 2>&1 |&
#
# fd 0 is standard-in, fd 1 is standard-out, and fd 2 is standard-error.
# The line "exec 4&>1" opens fd 4 and assigns it to whatever fd 1 was
# assigned to. I am sorta "saving a copy of fd 1 in fd 4".
#
# The line "ftp -nv >&4 2>&1 |&" is a little harder.
#
# The "|&" turns the process into a co-process that allows subsequent
# "print -p" statements to send lines to the co-process' standard-in and
# "read -p" to read from its standard-out. So ksh forks a copy of itself
# and fiddles with the fd's 0 and 1 until this it set-up. But it leaves
# the rest of the fd's alone.
#
# Then it encounters ">&4" which causes it to set the ftp process' standard
# out to whatever 4 is. Well since 4 is a copy of 1 before the co-process,
# we are back to writing to the original shell's standard out. Lastly,
# the 2>&1 does the same thing for standard error.
#
# CHANGE LOG
# =========================================================================
# 07/11/2006 - Goran Base script created
# 09/11/2006 - Goran Created usage function
# 24/11/2006 - Goran Added error checking of ftp transfers
# 27/11/2006 - Goran Added ftptest function
# 01/12/2006 - Goran Added cleanup function
# 25/10/2007 - Goran Added CheckDir function
#
LHOST=`uname -n`
RHOST=123.456.789.123
USER=Enter your userid here
PASSWD=Enter your password here
ADMIN=unix-admin@mydomain.com
TSTFILE=TEST.TXT
LOG=/var/adm/ftp.log
LOG1=/tmp/ftp.mail
LOG2=/tmp/ftp.log
LOG3=/tmp/ftp.parselog
LIN=/export/ftpdir/Incoming
LOUT=/export/ftpdir/Outgoing
RIN=/$USER/Incoming
ROUT=/$USER/Outgoing
usage ()
{
echo ""
echo ""
echo " #########################################################"
echo " ## ##"
echo " ###### Usage: autoftp [ mget | mput | xfer ] ######"
echo " ## ##"
echo " #########################################################"
echo " ## ##"
echo " ## mget: Copy files from $RHOST to $LHOST ##"
echo " ## ##"
echo " ## mput: Copy files from $LHOST to $RHOST ##"
echo " ## ##"
echo " ## xfer: Perform an mget and an mput from $LHOST ##"
echo " ## to $RHOST ##"
echo " ## ##"
echo " #########################################################"
echo ""
echo ""
}
SendMail()
{
SUBJECT=$1
OUTPUT=$2
cat $OUTPUT | mailx -s "$SUBJECT" $ADMIN
}
CheckDir()
{
DIR=$1
if [ ! -d $DIR ]; then
echo "Error - Directory $DIR does not exist" >> $LOG2
echo "ERROR: Directory Error - $DIR does not exist" | mailx -s "Directory Error - $DIR" $ADMIN
exit 1
fi
}
doftp ()
{
CMD=$1
LOCAL=$2
REMOTE=$3
if [[ $CMD = "mget" ]]; then
echo "================================================" >> $LOG2
echo "== Receiving Files From $RHOST ==" >> $LOG2
echo "================================================" >> $LOG2
else
echo "============================================" >> $LOG2
echo "== Sending Files to $RHOST ==" >> $LOG2
echo "============================================" >> $LOG2
fi
exec 4>&1
ftp -nv >&4 2>&1 |&
pid2=$!
print -p open $RHOST
print -p user $USER $PASSWD
print -p binary
print -p lcd $LOCAL
print -p cd $REMOTE
print -p prompt
print -p $CMD \*
print -p bye
wait $pid2
}
ftptest ()
{
mv $LIN/$TSTFILE $LOUT
echo "=======================================" >> $LOG2
echo "== $TSTFILE being sent back to $RHOST ==" >> $LOG2
echo "=======================================" >> $LOG2
exec 4>&1
ftp -nv >&4 2>&1 |&
pid4=$!
print -p open $RHOST
print -p user $USER $PASSWD
print -p binary
print -p lcd $LOUT
print -p cd $RIN/.receiving
print -p put $TSTFILE
print -p bye
wait $pid4
}
log ()
{
CMD=$1
SRC=$2
echo "=== $CMD Files Report ===" >> $LOG1
cd $SRC
echo "Size(KB) File" >> $LOG1
ls -s | grep -v total | awk '{ $size = $1 / 2; print $size " " $2 }' | sort -n >> $LOG1
echo "=============================" >> $LOG1
}
mvfiles ()
{
LOCAL=$1
LDEST=$2
REMOTE=$3
RDEST=$4
exec 4>&1
ftp -n >&4 2>&1 |&
pid3=$!
print -p open $RHOST
print -p user $USER $PASSWD
print -p binary
print -p cd $REMOTE
cd $LOCAL
for files in `ls` ; do
print -p rename $files $RDEST/$files
mv $files $LDEST
done
print -p bye
wait $pid3
}
chkerror ()
{
cat /dev/null > $LOG3
cat $LOG2 >> $LOG
grep -v bytes $LOG2 >> $LOG3
for error in `awk '{print $1}' $LOG3`; do
case $error in
Not) SendMail "FTP Error - between $LHOST and $RHOST" $LOG2
exit 1;;
332) SendMail "FTP Error - Need account for login" $LOG2
exit 1;;
350) SendMail "FTP Error - Requested file action pending further information" $LOG2
exit 1;;
421) SendMail "FTP Error - Service not available, closing control connection" $LOG2
#This may be a reply to any command if the service knows it
#must shut down.
exit 1;;
425) SendMail "FTP Error - Can't open data connection" $LOG2
exit 1;;
426) SendMail "FTP Error - Connection closed; transfer aborted" $LOG2
exit 1;;
450) SendMail "FTP Error - Requested file action not taken" $LOG2
#File unavailable (e.g., file busy).
exit 1;;
451) SendMail "FTP Error - Requested action aborted: local error in processing" $LOG2
exit 1;;
452) SendMail "FTP Error - Requested action not taken" $LOG2
#Insufficient storage space in system.
exit 1;;
500) SendMail "FTP Error - Syntax error, command unrecognized" $LOG2
#This may include errors such as command line too long.
exit 1;;
501) SendMail "FTP Error - Syntax error in parameters or arguments" $LOG2
exit 1;;
502) SendMail "FTP Error - Command not implemented" $LOG2
exit 1;;
503) SendMail "FTP Error - Bad sequence of commands" $LOG2
exit 1;;
504) SendMail "FTP Error - Command not implemented for that parameter" $LOG2
exit 1;;
530) SendMail "FTP Error - Not logged in" $LOG2
exit 1;;
532) SendMail "FTP Error - Need account for storing files" $LOG2
exit 1;;
550) SendMail "FTP Error - No such file or directory" $LOG2
#File unavailable (e.g., directory empty, no access).
exit 1;;
551) SendMail "FTP Error - Requested action aborted: page type unknown" $LOG2
exit 1;;
552) SendMail "FTP Error - Requested file action aborted" $LOG2
#Exceeded storage allocation (for current directory or
#dataset).
exit 1;;
553) SendMail "FTP Error - Requested action not taken - File name not allowed" $LOG2
exit 1;;
esac
done
}
cleanup ()
{
echo "=======================================" >> $LOG1
echo "=======================================" >> $LOG2
echo "== Cleaning up old sent files ==" >> $LOG1
echo "== Cleaning up old sent files ==" >> $LOG2
echo "=======================================" >> $LOG1
echo "=======================================" >> $LOG2
echo " Files removed from $LHOST:$LOUT/.sent" >> $LOG1
echo " Files removed from $LHOST:$LOUT/.sent" >> $LOG2
## Clean up local servers Outgoing folder
cd $LOUT/.sent
find $LOUT/.sent -mtime +14 >> $LOG1
find $LOUT/.sent -mtime +14 >> $LOG2
find $LOUT/.sent -mtime +14 -exec rm {} \;
## Create the .history folder
cat /dev/null > $LIN/.history/.list
cd $LIN
for history in `ls` ; do
touch $LIN/.history/$history
done
exec 4>&1
ftp -n >&4 2>&1 |&
pid5=$!
print -p open $RHOST
print -p user $USER $PASSWD
print -p binary
print -p cd $ROUT/.sent
cd $LIN/.history
find $LIN/.history -mtime +14 >> $LIN/.history/.list
for old in `cat $LIN/.history/.list` ; do
print -p delete $old # Delete file from remote server
rm $old # Delete file from local folder
done
print -p bye
wait $pid5
echo " Files removed from $RHOST:$ROUT/.sent" >> $LOG1
echo " Files removed from $RHOST:$ROUT/.sent" >> $LOG2
cat $LIN/.history/.list >> $LOG1
cat $LIN/.history/.list >> $LOG2
echo "" >> $LOG2
}
## Create log headers here
echo "=============================" > $LOG1
echo "=============================" > $LOG2
echo " `date`" >> $LOG1
echo " `date`" >> $LOG2
echo "=============================" >> $LOG1
echo "=============================" >> $LOG2
CheckDir $BASE
CheckDir $LIN
CheckDir $LOUT
## Create required local directories
test ! -d $LIN/.receiving && mkdir $LIN/.receiving
test ! -d $LIN/.history && mkdir $LIN/.history
test ! -d $LOUT/.sent && mkdir $LOUT/.sent
CheckDir $LIN/.receiving
CheckDir $LIN/.history
CheckDir $LOUT/.sent
exec 5>&1
# Switch the comments on the next 2 lines if
# you also want to see output on the screen
#tee /dev/tty >> $LOG2 |&
tee >> $LOG2 |&
pid1=$!
exec >&p 2>&1 4>&1
case $1 in
mget) doftp mget $LIN/.receiving $ROUT
log Recieved $LIN/.receiving
mvfiles $LIN/.receiving .. $ROUT .sent ;;
mput) doftp mput $LOUT $RIN/.receiving
log Sent $LOUT
mvfiles $LOUT .sent $RIN/.receiving .. ;;
xfer) doftp mget $LIN/.receiving $ROUT
log Recieved $LIN/.receiving
mvfiles $LIN/.receiving .. $ROUT .sent
doftp mput $LOUT $RIN/.receiving
log Sent $LOUT
mvfiles $LOUT .sent $RIN/.receiving .. ;;
*) usage
exec >&- 2>&- 4>&- >&5 2>&1
exit 1 ;;
esac
## Run ftptest if test file exists in remote Outgoing folder
if [[ -e "$LIN/$TSTFILE" ]]; then
ftptest $tstfile
mvfiles $LOUT .sent $RIN/.receiving ..
exec >&- 2>&- 4>&- >&5 2>&1
chkerror
SendMail "FTP Test Results - between $LHOST and $RHOST" $LOG2
exit 0
fi
exec >&- 2>&- 4>&- >&5 2>&1
## Remove sent files older then 14 days
## From local and remote servers
cleanup
chkerror
#Remove comment below to receive an email on successfull ftp
#SendMail "FTP Results - between $LHOST and $RHOST" $LOG1
exit 0
This is writen for a Solaris 8 system, thats why I have to do the ls -s and awk stuff and can't just use ls -h in my log function. I also need to remove all files older then 14 days from the .sent directory. This is easy on the server as I just use find, but its a bit more tricky on the remote Windows NT system.
UPDATED
I have updated the code above to include my cleanup function..
I haven't read the entire thread, but when it comes to automated FTP transfers ( every day to backup machine ) use rsync.
In my case rsync is blocked by 2 firewalls, But even if it wasn't blocked the customer doesn't want to install any additional software and wants to use their ftp server.
You will need to get a directory listing of the remote directory and parse it to extract the name and date. See this thread to see a script that does this sort of thing.
I managed to get it working by adding this function.
cleanup ()
{
echo "=======================================" >> $LOG2
echo "== Cleaning up old sent files ==" >> $LOG2
echo "=======================================" >> $LOG2
echo " Files removed from $LHOST:$LOUT/.sent" >> $LOG2
## Clean up local servers Outgoing folder
cd $LOUT/.sent
find $LOUT/.sent -mtime +14 >> $LOG2
find $LOUT/.sent -mtime +14 -exec rm {} \;
## Create the .history folder
cat /dev/null > $LIN/.history/.list
cd $LIN
for history in `ls` ; do
touch $LIN/.history/$history
done
exec 4>&1
ftp -n >&4 2>&1 |&
pid5=$!
print -p open $RHOST
print -p user $USER $PASSWD
print -p binary
print -p cd $ROUT/.sent
cd $LIN/.history
find $LIN/.history -mtime +14 >> $LIN/.history/.list
for old in `cat $LIN/.history/.list` ; do
print -p delete $old # Delete file from remote server
rm $old # Delete file from local folder
done
print -p bye
wait $pid5
echo " Files removed from $RHOST:$ROUT/.sent" >> $LOG2
cat $LIN/.history/.list >> $LOG2
echo "" >> $LOG2
}
I've updated the script I posted above to include the cleanup function..
Hi,
I tried the ftp program in cygwin and was not able to execute it. It shows error
--------------------------------------------
nortel_admin@TCS036694 /usr/local/bin
$ ksh ftpsh
ftpsh: 9: Syntax error: "&" unexpected
nortel_admin@TCS036694 /usr/local/bin
$
--------------------------------------------
basically it refers to line
*************************
exec 4>&1
ftp -nv >&4 2>&4 | &
*************************
Please advice...
Regards,
Sujit Menon
didn't get what it does.
please explain.
thanks
Before you run the script try changing to a ksh shell, instead of:
$ ksh ftpsh
try
$ exec ksh
$ ftpsh
I wrote the script using some functions from posts on these forums, that bit was something that Perderabo came up with I think... It is a bit confusing, thats why I added an explanation for it at the top of the script. It think thats a good explanation and I wouldn't know how to explain it any differently.
I'm using your code but i have a problem,
i need the code to be flexible in such a way that it changes the filename it gets everyday. And it should be done automatically.
For example,
1st day: get apr20.log
2nd day: get apr21.log
#! /usr/bin/ksh
HOST=remote.host.name
USER=whoever
PASSWD=whatever
exec 4>&1
ftp -nv >&4 2>&4 |&
print -p open $HOST
print -p user $USER $PASSWD
print -p cd directory
print -p binary
print -p get -filename that changes automatically everyday-
print -p bye
wait
exit 0