#!/bin/bash
# request Bourne shell as shell for job
#$ -S /bin/bash
# assume current working directory as paths
#$ -cwd
#$ -N rsync-copy
#
# print date and time
date
rsync -rltD --progress "ssh -i /home/myname/.ssh/id_rsa" /home/myname/Desktop/rsync-sample user@target.server.edu:~
# print date and time again
date
I did use key-gen to generate my public keys and had them in my target server's authorized keys. I can ssh without entering my password. But this particular when I submit it to the cluster using qsub, it just sits their hanging without doing anything.
My suspicion is that the "--progress" (marked bold) option needs a terminal to work and "ssh" - depending on its configuration - might not necessarily provide one.
You might try adding a "-t" to the ssh-options, which forces the allocation of a pseudo-tty. See "man ssh" for details about this.
I did remove the --progress option and added the -t option to my ssh.
But still the job hangs with a status of t and no data is transferred across servers.
But just simply running the rsync command at the prompt works absolutely fine. But running in a bash script is much more easier for me because I have around 55TB of data that I need to copy.
I have no idea why this is so either, but you could perhaps execute proper debugging procedures: start by adding "set -xv" in your script like this:
#! /bin/bash
set -xv
[... rest of your script...]
set +xv
exit 0
which switches on a lot of diagnostic shell output (see the man page of "bash" for details). Capture this shell output into a file and analyse every line carefully.