I would like to scp multiple files to a remote server that requires a password for the process to be completed.
I have 30 folders ( x_1 , x_2 ... x_30 ), each containing 25 files. What I want to do is scp 1 out of the 25 files (file called bvals ) for all my folders to a remote server and place each file to its respective folder ( x_1 , x_2 ... x_30 in the remote server).
I wrote this script:
for n in x_1 x_2 x_3 x_4 x_5 x_6 x_7 x_8 x_9 x_10 x_11 x_12 x_13 x_14 x_15 x_16 x_17 x_18 x_19 x_20 x_21 x_22 x_23 x_24 x_25 x_26 x_27 x_28 x_29 x_30
do
scp $HOME/data/${n}/bvals remote_server@remote.edu:/me/my_data/${n}
done
The problem with this script (of course) is that every time n takes a new argument ( x_1 etc) I am asked to insert my password.
Is there any way to write this script in a way such that I am prompted to write my password only once?
# create a single tar file , scp the one file to the remote
cd /path/my/directories # local place where directories are
tar cvf myfiles.tar x_1 x_2 x_3 x_4 x_5 x_6 x_7 x_8 x_9 x_10 x_11 x_12 x_13 x_14 x_15 x_16 x_17 x_18 x_19 x_20 x_21 x_22 x_23 x_24 x_25 x_26 x_27 x_28 x_29 x_30
# scp tar file onto the remote box
scp myfiles.tar remote_server@remote.edu:/me/my_data/
# extract all of the directories to the place they belong
ssh remote_server@remote.edu 'cd remote_server@remote.edu && tar xf myfiles.tar
If the tar file is gigantic you may want to compress it first, send it, uncompress on the other side, extract the new re-sized tar file.
There is actually another way to move the files directly, more complicated but for long transmissions runs into issues with crummy connections. This is my plan A when I know enough to feel okay about moving lots of data. I gave you plan B because it is less likely to bomb from a dropped connection.
The reason for this is that some ISP's in my area 're-establish' DHCP periodically, dropping the old connection.
Example:
cd /path/to/local/directories
tar cvf - . | ssh -e none me@remote 'cd /path/to/remote && ( tar xf - )'
As an aside from the actual script: is it possible you could use SSH key-based authentication, rather than passwords ? This would enable you to connect without having to supply a password, with authentication taking place by means of a key exchange. This way you could transfer as many files as you liked as often as you liked without having to type in anything (though doing it in as few transfers as possible as per the solution you've been given is definitely a better way to go regardless).
If the source and destination servers are running OpenSSH, then the idea is that you would first generate a DSA or RSA public key for yourself. You might in fact find that you've got one or both of these already, stored as '.ssh/id_dsa.pub' or '.ssh/id_rsa.pub' underneath your home directory. Next, you must add the contents of your public key file to '.ssh/authorized_keys' in the home directory of the receiving account on the destination server.
After doing that you should be able to log in without being prompted for a password at all, so long as the permissions on all appropriate files and directories are secure enough at both ends. Essentially they must be accessible only by the user to whom the account belongs, more or less. Just a thought, if that's an option for you.