automate sftp using unix script

Did you try changing the "hit-dw" with your local server name.

Duh, dummy me. Yes, after I posted a reply I realized what I should have done. I added the ip address of the secure server to ftpsvr and the name of the secure server for hit-dw. I also tried using the server I was ftping from for ftpsvr and that didn't work either. I got the following error message:

ftp: connect: Connection refused
Not connected.
Not connected.
Not connected.
Not connected.

Even i'm also getting the same ftp: connect: Connection refused error when i'm trying out the below code

HOSTNAME=xx.xxx.xx.xxx
USER=username
PSSWD=password
datedir=`TZ=CST+48 date +%m%d%Y`
REP_SRC=/path/path1/$datedir
REP_DEST=/path2/dir
FILENAME=abc.txt

rm -f log_file.tmp

ftp -i -n -v $HOSTNAME> .ftpout$$<<EOF
open $HOSTNAME
user $USER $PSSWD
ascii
cd "$REP_SRC"
pwd
lcd "$REP_DEST"
get $FILENAME TEST_abc.txt
bye
EOF

While i'm trying to execute the same code for different remote machine its working fine. But in particular with one machine it isn't working ...

Do anyone know why its behaving like that ???

Today I had the same problem, that I need to do an sftp connection for a script that will not run in interactive mode.

#!/bin/sh

HOST=XX.XXXX.XXX
USER=yourusername
PASS=yourpassword

echo "Starting to sftp..."

lftp -u ${USER},${PASS} sftp://${HOST} <<EOF
cd somedir
get somefile
bye
EOF

echo "done"

Note: lftp needs to get told which protocol to use. The above example works for sftp.

Not sure if this info is still desired, but I have just finished an auto sftp script.
All servers run a script to backup files, then this script runs to gather all the output files and centralize.
ssh keys must be set up and working for script to work. hope it helps.

#!/bin/ksh

HOSTS="svr1 svr2 svr3 svr4 svr5"
DATE=`date '+%m%d%y.%H%M'`
LOG=/usr/local/scripts/xfer_failed

export HOSTS DATE LOG

### Main ###

if [ -f $LOG ];
 then mv $LOG $LOG.$DATE
 touch $LOG
 else touch $LOG
fi
#
for each in $HOSTS;
do
  if [ ! -d /usr/restore/$each ];
    then mkdir /usr/restore/$each
  fi
cd /usr/restore/$each
find . -type f -name "$each*.tar*" -mtime +14 -exec rm {} \;
#
sftp -b - $each << EOF
get /root/restore/$each.tar $each.$DATE.tar
bye
EOF
  if [ $? -ne 0 ];
    then echo "File transfer for $each failed. $DATE" >> $LOG
    cat $LOG | mailx -s "Vitalfile transfer failure" xxxxxxx@xxxxxx.com
    else continue
  fi
done

find /usr/local/scripts -type f -name "xfer_failed.*" -mtime +4 -exec rm {} \;

### End Main ###

Hi,

I know some of you are looking for a solution on automating sftp using expect to work with Cron. I've been searching for the solution to thi sissue for weeks now. I've finally found the solution and thought I should share.

first I used autoexpect to generate the script from my sftp session, then i modified it to read the password from a file.

autoexpect sftp -b Batchfile username@sftpserver

here is the completed script. It works with cron also.

#!/usr/bin/expect -f
#
set force_conservative 0  ;# set to 1 to force conservative mode even if
                          ;# script wasn't run conservatively originally
if {$force_conservative} {
        set send_slow {1 .1}
        proc send {ignore arg} {
                sleep .1
                exp_send -s -- $arg
        }
}

set authFile "Path to password file"
if {![file exists $authFile]} {;
send_user "$authFile does not exist; aborting\n"
exit 1
}
set fileFD [open $authFile r];
gets $fileFD authLine;
close $fileFD

set timeout -1
spawn /usr/bin/sftp -b batchfile username@sftpserver.com
match_max 100000
# note: the autoexpect command will produce the line below. helpful if the #sftpserver has a long login banner.
expect -exact "user@sftpserver.com\r
send -- "$authLine\r"
expect eof

hope this helps. it works for me. I call this script from a batch file .

Hi,

I did the same as you did. but what is the reason why i can't make it work? btw, this is my script.

if {$force_conservative} {
        set send_slow {1 .1}
        proc send {ignore arg} {
                sleep .1
                exp_send -s -- $arg
        }
}

set timeout -1
spawn /usr/bin/sftp user1@server1
match_max 100000
expect -exact "Connecting to server1...\r
user1@server1's password: "
send -- "hardcode_password\r"
expect -exact "\r
sftp> "
send -- "cd .ssh\r"
expect -exact "cd .ssh\r
sftp> "
send -- "get file1\r"
expect -exact "get file1\r
Fetching /users/user1/.ssh/file1 to file1\r
sftp> "
send -- "exit\r"
expect eof

i think the only difference we had is how the password file was handled. what is the $authfile in your script?

thanks

Hi All,

I'm attempting the same thing here; my particular problem is that I'm trying to get files from a UNIX box to a VMS box with different account names. So, I believe, the shared public key option wouldn't work for me (because the account names have to be the same?).

Anyway, I'm interested in the 'expect' solutions posted by the previous participants of this thread but they don't seem to work for me. In the last post there's a variable $arg mentioned - is this the password? And is this passed on the command line?

Many thanks,

p.

Hi
I have to write script whic will do ssh login to Remote server with out password and then secure file copy to that remote server.
and i want that to be done automatically(means no prompt should be displyed)

can u please help me in this for automation
Thanks

Hello All,

I figured Id save people a whole bunch of problems with this process. When you automate SFTP using expect the "-b" flag WILLNOT work under crontab due to differences in the enviroment. You must wrap all the sftp commands that you want to use inside the expect script and axe the batch script. If you tell it to run off of a batch script it will work running manually however not through cron.

I read the artcile and observed that this is between a local unix to another local unix machine. What is the procedure if the remote server is windows????

I too am attempting to write a shell script using expect to automate a file transfer from a Windows box to a Sun box.

I have eliminated the -B option and have hard coded all prompts and replies in my expect script and it actually runs via cron however I am only getting a partial download.

Here is the expect script:

#!/usr/local/bin/expect -f -d
spawn  sftp -v ftplogin@ftpserver
expect "ftplogin@ftpserver's password: "
sleep 7
send "thepassword\r";
expect "sftp>"
send "lcd /tmp\r"
expect "sftp>"
send "get dailyinventory.txt\r"
expect "sftp>"
send "exit\r"

Here is a sample of the log created by the -v option

spawn sftp -v ftpuser@ftpserver
Connecting to ftpserver...
Sun_SSH_1.1, SSH protocols 1.5/2.0, OpenSSL 0x0090700f
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: Rhosts Authentication disabled, originating port will not be trusted.
debug1: ssh_connect: needpriv 0
debug1: Connecting to ftpserver [ftpserver] port 22.
debug1: Connection established.
debug1: identity file /.ssh/id_rsa type -1
debug1: identity file /.ssh/id_dsa type 2
debug1: Remote protocol version 2.0, remote software version WRQReflectionForSecureIT_6.1 Build 21
debug1: no match: WRQReflectionForSecureIT_6.1 Build 21
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-Sun_SSH_1.1
debug1: Failed to acquire GSS-API credentials for any mechanisms (No credentials were supplied, or the credentials were unavailable or inaccessible
mech_dh: Invalid or unknown error
)
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: server->client aes128-cbc hmac-md5 none
debug1: kex: client->server aes128-cbc hmac-md5 none
debug1: Peer sent proposed langtags, ctos: 
debug1: Peer sent proposed langtags, stoc: 
debug1: We proposed langtags, ctos: i-default
debug1: We proposed langtags, stoc: i-default
debug1: dh_gen_key: priv key bits set: 131/256
debug1: bits set: 495/1024
debug1: sending SSH2_MSG_KEXDH_INIT
debug1: expecting SSH2_MSG_KEXDH_REPLY
debug1: Host 'ftpserver' is known and matches the DSA host key.
debug1: Found key in /.ssh/known_hosts:5
debug1: bits set: 528/1024
debug1: ssh_dss_verify: signature correct
debug1: newkeys: mode 1
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: newkeys: mode 0
debug1: SSH2_MSG_NEWKEYS received
debug1: done: ssh_kex2.
debug1: send SSH2_MSG_SERVICE_REQUEST
debug1: got SSH2_MSG_SERVICE_ACCEPT
debug1: Authentications that can continue: publickey,password
debug1: Next authentication method: publickey
debug1: Trying private key: /.ssh/id_rsa
debug1: Trying public key: /.ssh/id_dsa
debug1: Authentications that can continue: publickey,password
debug1: Next authentication method: password
ftpuser@ftpserver's password: 
debug1: Authentication succeeded (password)
debug1: fd 6 setting O_NONBLOCK
debug1: channel 0: new [client-session]
debug1: send channel open 0
debug1: Entering interactive session.
debug1: ssh_session2_setup: id 0
debug1: Sending subsystem: sftp
debug1: channel request 0: subsystem
debug1: channel 0: open confirm rwindow 100000 rmax 32768
sftp> lcd /tmp
sftp> get dailyinventory.txt
Fetching /HOME/dailyinventory.txt to dailyinventory.txt

The problem is that the dailyinventory.txt file download has 16,586 records but the dailyinventory.txt file on the ftpserver has 68,082 records.

Wondering if anyone else has seen this problem w/ partial downloads?

Thanks in advance,

pat

you can mount the win FS using SAMBA and working like it has unix

Hi,
I need to write a script which PULLs files from UNIX server (Solaris) to another UNIX box (Solaris). My requirements are as follows:
(1) I dont want to generate key pairs coz I would be having atleast 50 remote servers from which I need to PULL files
(2) No clear-text user/name passwords
(3) I was looking for NET::SFTP using perl scripting but there are some issues installing the NET::SFTP module currently

Can anyone help me out here with some other option to PULL files securely?

Thanks!

Hi

Did anyone find a solution to this? When automate sftp using a script it still expects a password, which i have to type in.

My script is simple

#!/bin/sh
sftp username@servername << end
        passowrd
        cd directoryname 
        put filename
        bye
end

Can anyone please help
Dinesh Patel