Ssh connection timed out

omonoiatis9 · December 13, 2012, 1:28am

Hello,

I have two AIX6.1 machines that can communicate with each other through ssh.
The problem is that one of them somehow closes the connection after some time and i don't know the reason of that.
For example today i send through rsync command 7 files from one server to the other. It send successfully the first 6 files but the last one failed with the following error:

ssh: connect to host <server_name> port 22: Connection timed out
rsync: connection unexpectedly closed (0 bytes read so far)
rsync error: error in rsync protocol data stream (code 12) at io.c(342)
FAILURE <file_name> at SERVER <server_name>

I never experienced this kind of problem before. Why it loses the connection?

Thank you for your attention

vpundit · December 13, 2012, 1:18pm

omonoiatis9:

Hello,

I have two AIX6.1 machines that can communicate with each other through ssh.
The problem is that one of them somehow closes the connection after some time and i don't know the reason of that.
For example today i send through rsync command 7 files from one server to the other. It send successfully the first 6 files but the last one failed with the following error:
ssh: connect to host <server_name> port 22: Connection timed out
rsync: connection unexpectedly closed (0 bytes read so far)
rsync error: error in rsync protocol data stream (code 12) at io.c(342)
FAILURE <file_name> at SERVER <server_name>
I never experienced this kind of problem before. Why it loses the connection?

Thank you for your attention

was there any changes to your sshd_conf?

check to see if you have the TCPKeepAlive active.

Also have you tried to restart the service? and try using a

ssh -vv

kah00na · December 13, 2012, 2:23pm

I'm not sure how rsynch works, but I do know that the SSH server is limited to only have a certain number of unauthenticated SSH connections at one time. What is your "MaxStartups" set to in your /etc/ssh/sshd_config file? My guess is that it is set to 6.

hostname:/:$ grep MaxStartups /etc/ssh/sshd_config
MaxStartups 15
hostname:/:$

If my guess proves right, bump it up to 15 or so, restart the sshd (stopsrc -s sshd && startsrc -s sshd), and try it again.

omonoiatis9 · December 14, 2012, 4:41am

i check both parameters that you mentioned.
MaxStartups is up to 1000 so i guess that it should not be a problem.
TCPKeepAlive is set to yes but i see that is commented out.
Shall i enable it?
if you think that the output from ssh -vv will be helpful here it is:

OpenSSH_5.8p1, OpenSSL 0.9.8r 8 Feb 2011
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: Failed dlopen: /usr/krb5/lib/libkrb5.a(libkrb5.a.so):   0509-022 Cannot load module /usr/krb5/lib/libkrb5.a(libkrb5.a.so).
        0509-026 System error: A file or directory in the path name does not exist.

debug1: Error loading Kerberos, disabling Kerberos auth.
debug2: ssh_connect: needpriv 0
debug1: Connecting to rea [208.5.3.46] port 22.
debug1: Connection established.
debug1: permanently_set_uid: 0/0
debug2: key_type_from_name: unknown key type '-----BEGIN'
debug2: key_type_from_name: unknown key type '-----END'
debug1: identity file /.ssh/id_rsa type 1
debug1: identity file /.ssh/id_rsa-cert type -1
debug2: key_type_from_name: unknown key type '-----BEGIN'
debug2: key_type_from_name: unknown key type '-----END'
debug1: identity file /.ssh/id_dsa type 2
debug1: identity file /.ssh/id_dsa-cert type -1
debug1: Remote protocol version 2.0, remote software version OpenSSH_5.8
debug1: match: OpenSSH_5.8 pat OpenSSH*
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_5.8
debug2: fd 4 setting O_NONBLOCK
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug2: kex_parse_kexinit: diffie-hellman-group-exchange-sha256,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1
debug2: kex_parse_kexinit: ssh-rsa-cert-v01@openssh.com,ssh-rsa-cert-v00@openssh.com,ssh-rsa,ssh-dss-cert-v01@openssh.com,ssh-dss-cert-v00@openssh.com,ssh-dss
debug2: kex_parse_kexinit: aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour,rijndael-cbc@lysator.liu.se
debug2: kex_parse_kexinit: aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour,rijndael-cbc@lysator.liu.se
debug2: kex_parse_kexinit: hmac-md5,hmac-sha1,umac-64@openssh.com,hmac-ripemd160,hmac-ripemd160@openssh.com,hmac-sha1-96,hmac-md5-96
debug2: kex_parse_kexinit: hmac-md5,hmac-sha1,umac-64@openssh.com,hmac-ripemd160,hmac-ripemd160@openssh.com,hmac-sha1-96,hmac-md5-96
debug2: kex_parse_kexinit: none,zlib@openssh.com,zlib
debug2: kex_parse_kexinit: none,zlib@openssh.com,zlib
debug2: kex_parse_kexinit:
debug2: kex_parse_kexinit:
debug2: kex_parse_kexinit: first_kex_follows 0
debug2: kex_parse_kexinit: reserved 0
debug2: kex_parse_kexinit: diffie-hellman-group-exchange-sha256,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1
debug2: kex_parse_kexinit: ssh-rsa,ssh-dss
debug2: kex_parse_kexinit: aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour,rijndael-cbc@lysator.liu.se
debug2: kex_parse_kexinit: aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour,rijndael-cbc@lysator.liu.se
debug2: kex_parse_kexinit: hmac-md5,hmac-sha1,umac-64@openssh.com,hmac-ripemd160,hmac-ripemd160@openssh.com,hmac-sha1-96,hmac-md5-96
debug2: kex_parse_kexinit: hmac-md5,hmac-sha1,umac-64@openssh.com,hmac-ripemd160,hmac-ripemd160@openssh.com,hmac-sha1-96,hmac-md5-96
debug2: kex_parse_kexinit: none,zlib@openssh.com
debug2: kex_parse_kexinit: none,zlib@openssh.com
debug2: kex_parse_kexinit:
debug2: kex_parse_kexinit:
debug2: kex_parse_kexinit: first_kex_follows 0
debug2: kex_parse_kexinit: reserved 0
debug2: mac_setup: found hmac-md5
debug1: kex: server->client aes128-ctr hmac-md5 none
debug2: mac_setup: found hmac-md5
debug1: kex: client->server aes128-ctr hmac-md5 none
debug1: SSH2_MSG_KEX_DH_GEX_REQUEST(1024<1024<8192) sent
debug1: expecting SSH2_MSG_KEX_DH_GEX_GROUP
debug2: dh_gen_key: priv key bits set: 124/256
debug2: bits set: 482/1024
debug1: SSH2_MSG_KEX_DH_GEX_INIT sent
debug1: expecting SSH2_MSG_KEX_DH_GEX_REPLY
debug1: Server host key: RSA 22:55:96:bc:9b:45:f3:7b:0d:55:98:73:f4:01:e0:4f
debug1: Host 'rea' is known and matches the RSA host key.
debug1: Found key in /.ssh/known_hosts:50
debug2: bits set: 499/1024
debug1: ssh_rsa_verify: signature correct
debug2: kex_derive_keys
debug2: set_newkeys: mode 1
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug2: set_newkeys: mode 0
debug1: SSH2_MSG_NEWKEYS received
debug1: Roaming not allowed by server
debug1: SSH2_MSG_SERVICE_REQUEST sent
debug2: service_accept: ssh-userauth
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug2: key: /.ssh/id_rsa (20061888)
debug2: key: /.ssh/id_dsa (200619f8)
debug1: Authentications that can continue: publickey,password,keyboard-interactive
debug1: Next authentication method: publickey
debug1: Offering RSA public key: /.ssh/id_rsa
debug2: we sent a publickey packet, wait for reply
debug1: Authentications that can continue: publickey,password,keyboard-interactive
debug1: Offering DSA public key: /.ssh/id_dsa
debug2: we sent a publickey packet, wait for reply
debug1: Server accepts key: pkalg ssh-dss blen 435
debug2: input_userauth_pk_ok: fp e0:0f:ae:5b:23:cb:72:6a:0a:35:d1:68:2b:67:af:8b
debug1: read PEM private key done: type DSA
debug1: Authentication succeeded (publickey).
Authenticated to rea ([208.5.3.46]:22).
debug1: channel 0: new [client-session]
debug2: channel 0: send open
debug1: Requesting no-more-sessions@openssh.com
debug1: Entering interactive session.
debug2: callback start
debug2: client_session2_setup: id 0
debug2: fd 4 setting TCP_NODELAY
debug2: channel 0: request pty-req confirm 1
debug2: channel 0: request shell confirm 1
debug2: callback done
debug2: channel 0: open confirm rwindow 0 rmax 32768
debug2: channel_input_status_confirm: type 99 id 0
debug2: PTY allocation request accepted on channel 0
debug2: channel 0: rcvd adjust 2097152
debug2: channel_input_status_confirm: type 99 id 0
debug2: shell request accepted on channel 0

i just cannot see anything that mentions limit or something like that

vpundit · December 14, 2012, 10:59am

before you go about enabling the feature. when was the last time you restarted the service?

stopsrc -s sshd 
startsrc -s sshd

Best to try and restart the service before you play around in the sshd_confiig

omonoiatis9 · December 17, 2012, 1:00am

i already tried that.
i stopped and start the subsystem again the first time it behave like that.
but it doesn't seem to make any difference.

kah00na · December 17, 2012, 9:25am

Do you have "auth.debug" enabled in your /etc/syslog.conf? If not, configure it and look in the logs and look for any errors in it. Here is my auth.debug line I use in my /etc/syslog.conf:

auth.debug      /var/log/syslog.auth.debug      rotate size 5m files 1

omonoiatis9 · December 17, 2012, 11:27am

i already have it configured but it doesn't write anything to it