automate sftp using unix script

priyamurthy2005 · April 1, 2005, 9:24am

Hi All,
I need to write a UNIX script that automates the sftp process. I should be able to do a sftp to a secure box and get a file from there. I am having a problem doing this because no matter what I do, when I run my script, I get a prompt at command line asking for a password. How could I automate this? Could someone give me a sample script?
Thanks in advance.
Priya

dangral · April 1, 2005, 11:20am

This has been answered before in the forums here.
If you don't want to specify a password you need to generate a passwordless key with ssh-keygen and then copy the public key to the target host's ~/.ssh/authorized_keys file. This also has been discussed here before. Try a search.

priyamurthy2005 · April 1, 2005, 11:40am

Thank you so much dangral. I looked at your reply and looked at the sites you suggested. I am still at a loss. It asks me to use ssh. I tried that and created a public key file. Now it asks me for passphrase for public key file. I do not want any interactive mode at all. I somehow want it to automatically take the password for a file or something. I tried the following:

sftp -b passwordfile username@IP

The file "passwordfile" has passowrd stored in it. Still, it does not work. It first asks me for password interactively and only after I typed the password, it will execute the "passwordfile".
I looked on help for sftp and I was told that if I use non-interactive authentication, only then the above statement will work, meaning, it will take password for the batch file.
My question now is that: how do I achieve non-interactive authentication?

Thanks in advance.

dangral · April 1, 2005, 11:56am

I think you have two issues here:

1) Achieving passwordless authentication
2) automating sftp

I will try to discuss one at a time. Forgetting about sftp for a second, in order to achieve passwordless authentication ( in your terms non-interactive authentication) you need to set up a key that does not have a password. See this link for tips on how to set that up.

However, please see this warning and tips on how to set it up to be secure. (Sorry for all the links)

Once you have verified passwordless authetication is working, you can now attempt to automate sftp. What the link in my previous post said is:

You would not be putting a password file after sftp -b, you would be putting your standard-in device or a file with all the ftp commands. So on solaris, your script might look like this:

  
  #!/bin/sh
  echo "OK, starting now..."
  sftp -b /dev/fd/0 remotehost <<EOF
  cd pub
  ascii
  get filename.txt
  bye
  EOF

priyamurthy2005 · April 1, 2005, 12:37pm

Thanks again. Unfortunately, the command ssh-keygen does not work on my machine. It gives me the error:
ssh-keygen not found.
I guess I need to call my system admin and maybe get something installed, right???

RishiPahuja · April 20, 2005, 3:46am

dangral:

I think you have two issues here:

1) Achieving passwordless authentication
2) automating sftp

I will try to discuss one at a time. Forgetting about sftp for a second, in order to achieve passwordless authentication ( in your terms non-interactive authentication) you need to set up a key that does not have a password. See this link for tips on how to set that up.

However, please see this warning and tips on how to set it up to be secure. (Sorry for all the links)

Once you have verified passwordless authetication is working, you can now attempt to automate sftp. What the link in my previous post said is:

You would not be putting a password file after sftp -b, you would be putting your standard-in device or a file with all the ftp commands. So on solaris, your script might look like this:
  
  #!/bin/sh
  echo "OK, starting now..."
  sftp -b /dev/fd/0 remotehost <<EOF
  cd pub
  ascii
  get filename.txt
  bye
  EOF

Hi,
When I tried to use the snippet of your code, I am unable to connect. the -b option used by you is not viable in solaris. here it stands for buffersize.

> sftp -h
Usage: sftp2 [-D debug_level_spec] [-B batchfile] [-S path] [-h]
             [-V] [-P port] [-b buffer_size] [-N max_requests]
             [-c cipher] [-m mac] [-4] [-6] [-o option_to_ssh2]
             [user@]host[#port]
> sftp -V
sftp: F-Secure SSH 3.1.0 (build 12) on sparc-sun-solaris2.8
>

Thanks
Rishi

dangral · April 20, 2005, 4:32pm

You are using F-Secure sftp. The version I was referring to is OpenSSH.
Try using -B instead of -b.

RishiPahuja · April 21, 2005, 9:34am

Thanks Dangral

csaunders · June 15, 2005, 9:58am

dangral:

I think you have two issues here:

1) Achieving passwordless authentication
2) automating sftp

I will try to discuss one at a time. Forgetting about sftp for a second, in order to achieve passwordless authentication ( in your terms non-interactive authentication) you need to set up a key that does not have a password. See this link for tips on how to set that up.

However, please see this warning and tips on how to set it up to be secure. (Sorry for all the links)

Once you have verified passwordless authetication is working, you can now attempt to automate sftp. What the link in my previous post said is:

You would not be putting a password file after sftp -b, you would be putting your standard-in device or a file with all the ftp commands. So on solaris, your script might look like this:
  
  #!/bin/sh
  echo "OK, starting now..."
  sftp -b /dev/fd/0 remotehost <<EOF
  cd pub
  ascii
  get filename.txt
  bye
  EOF

the link you have given does not work, tried the steps on solaris 9 and it still asks for password....

STiVo · June 15, 2005, 2:24pm

Well if you can get the user keys working that will make things much easier.

Also, why don't you just use scp instead? Or maybe curl?

Something like this...

#!/bin/sh

scp remoteuser@remotehost:/path/to/file.txt /path/to/local/target

With the user keys setup you wouldn't have to interact with it at all- but remember it's a lot harder to get a password from my head than from my .ssh folder...

cyg · July 29, 2005, 12:40pm

using expect can get around the password verification issue.

/usr/local/bin/expect <<EOF
spawn  /usr/local/bin/sftp user@remotehost
expect "password:"
send "cd pub \r"
expect "sftp> "
send "ascii \r"
expect "sftp> "
send "get filename.txt \r"
expect "sftp> "
send "bye \r"
EOF

Phantom_Phixer · December 12, 2005, 11:29am

I am looking for a script that will allow me to automate file transfer via 'sftp'. key-gen is not an option as the 'client' (where we are 'getting' the files from) will mot permit us to generate the public/provate' keys for scp.
I have tried a number of examples shown and none of them work with sftp Each one either balks at the password or prompts for the password. Can someone show me a way to do this? Thanks.

LindaM · January 25, 2006, 10:01am

DID YOU EVER GET THIS TO WORK??

Vikas_Sood · May 8, 2006, 3:16pm

All,

This is a great forum for beginners like me. I have been following this thread in its multiple forms on various websites. I got a good start by following this thread but got stuck after doing exactly what has been mentioned in this thread word by word. The pain area : it was still prompting me for a password everytime.

Then, somebody suggested to recreate $HOME/.ssh directory again from scratch. Guess what, i did that and now i am able to log on from one server to another without being prompted for a password.

I dont know how much this information is gonna be useful to ppl but sure proved the definite solution for me and i thought i probably should share it with everybody.

Regards
Vikas.

unxuser · July 28, 2006, 12:39pm

Hi Vikas,

Would you please give us more details what mothed you used get around? Thanks.

neveryman · August 1, 2006, 9:14am

I struggled with this problem for quite a while before discovering lftp.

For Solaris you can get a copy at SunFreeware.

It allows you to use a shell type script with a password embedded for those servers that do not allow you to place a public key on them for passwordless authentication.

The "mirror" option is very handy indeed. Much easier to use than expect.

yvonneb · August 25, 2006, 1:13pm

I am also trying to automate sftp thru a cron file to be run every evening. My question is if I generate the key on the secure server and move it over to my server, will it affect other users from other servers. In other words can they still use their password to get into the secure server, or would they too have to copy a key over their server.

amitgupta_itbhu · October 17, 2006, 6:55am

Hi Priya could u please share that script for automating SFTP with all of us.

sbasetty · October 17, 2006, 10:59am

Can you try this and let me know

ftpsrv='XX.XX.XXX.XXX'
uid=myuser
passwd=mypass
ftp -n hit-dw  << endftp
user $uid $passwd
cd /file_folder
bye
endftp

yvonneb · October 17, 2006, 11:25am

I too am interested in this thread. I tried what you recommended and I received this error message:

ftp: Unknown Host hit-dw
Name does not resolve to supplied parameters; neither nodename nor servname were passed

.

Thanx,