SSH slow at connect

Hi experts,

We are getting slow ssh session connections at HP-UX 11.31 servers.

We have set the parameters that maybe will affect , and commented at other theads at config file sshd_config :

UseDNS no
X11Forwarding yes
X11DisplayOffset 10
X11UseLocalhost no
GSSAPIAuthentication no.

but these parameters no solve the error.

We've put the debug mode and the connection is waitting 10 seconds at ######..

ssh -v -v -v root@127.0.0.1
OpenSSH_5.6p1+sftpfilecontrol-v1.3-hpn13v7, OpenSSL 0.9.8o 01 Jun 2010
HP-UX Secure Shell-A.05.60.003, HP-UX Secure Shell version
debug1: Reading configuration data /opt/ssh/etc/ssh_config
debug3: RNG is ready, skipping seeding
debug2: ssh_connect: needpriv 0
debug1: Connecting to 127.0.0.1 [127.0.0.1] port 22.
debug1: Connection established.
debug1: permanently_set_uid: 0/3
debug1: identity file /.ssh/identity type -1
debug1: identity file /.ssh/identity-cert type -1
debug1: identity file /.ssh/id_rsa type -1
debug1: identity file /.ssh/id_rsa-cert type -1
debug1: identity file /.ssh/id_dsa type -1
debug1: identity file /.ssh/id_dsa-cert type -1
debug1: Remote protocol version 2.0, remote software version OpenSSH_5.6p1+sftpfilecontrol-v1.3-hpn13v7
debug1: match: OpenSSH_5.6p1+sftpfilecontrol-v1.3-hpn13v7 pat OpenSSH*
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_5.6p1+sftpfilecontrol-v1.3-hpn13v7
debug2: fd 4 setting O_NONBLOCK
debug3: RNG is ready, skipping seeding
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: AUTH STATE IS 0
debug2: kex_parse_kexinit: diffie-hellman-group-exchange-sha256,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1
debug2: kex_parse_kexinit: ssh-rsa-cert-v01@openssh.com,ssh-dss-cert-v01@openssh.com,ssh-rsa-cert-v00@openssh.com,ssh-dss-cert-v00@openssh.com,ssh-rsa,ssh-dss
debug2: kex_parse_kexinit: aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour,rijndael-cbc@lysator.liu.se
debug2: kex_parse_kexinit: aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour,rijndael-cbc@lysator.liu.se
debug2: kex_parse_kexinit: hmac-md5,hmac-sha1,umac-64@openssh.com,hmac-ripemd160,hmac-ripemd160@openssh.com,hmac-sha1-96,hmac-md5-96
debug2: kex_parse_kexinit: hmac-md5,hmac-sha1,umac-64@openssh.com,hmac-ripemd160,hmac-ripemd160@openssh.com,hmac-sha1-96,hmac-md5-96
debug2: kex_parse_kexinit: none,zlib@openssh.com,zlib
debug2: kex_parse_kexinit: none,zlib@openssh.com,zlib
debug2: kex_parse_kexinit:
debug2: kex_parse_kexinit:
debug2: kex_parse_kexinit: first_kex_follows 0
debug2: kex_parse_kexinit: reserved 0
debug2: kex_parse_kexinit: diffie-hellman-group-exchange-sha256,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1
debug2: kex_parse_kexinit: ssh-rsa,ssh-dss
debug2: kex_parse_kexinit: aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour,rijndael-cbc@lysator.liu.se
debug2: kex_parse_kexinit: aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour,rijndael-cbc@lysator.liu.se
debug2: kex_parse_kexinit: hmac-md5,hmac-sha1,umac-64@openssh.com,hmac-ripemd160,hmac-ripemd160@openssh.com,hmac-sha1-96,hmac-md5-96
debug2: kex_parse_kexinit: hmac-md5,hmac-sha1,umac-64@openssh.com,hmac-ripemd160,hmac-ripemd160@openssh.com,hmac-sha1-96,hmac-md5-96
debug2: kex_parse_kexinit: none,zlib@openssh.com
debug2: kex_parse_kexinit: none,zlib@openssh.com
debug2: kex_parse_kexinit:
debug2: kex_parse_kexinit:
debug2: kex_parse_kexinit: first_kex_follows 0
debug2: kex_parse_kexinit: reserved 0
debug2: mac_setup: found hmac-md5
debug1: REQUESTED ENC.NAME is 'aes128-ctr'
debug1: kex: server->client aes128-ctr hmac-md5 none
debug2: mac_setup: found hmac-md5
debug1: REQUESTED ENC.NAME is 'aes128-ctr'
debug1: kex: client->server aes128-ctr hmac-md5 none
debug1: SSH2_MSG_KEX_DH_GEX_REQUEST(1024<1024<8192) sent
debug1: expecting SSH2_MSG_KEX_DH_GEX_GROUP
debug2: dh_gen_key: priv key bits set: 115/256
debug2: bits set: 509/1024
debug1: SSH2_MSG_KEX_DH_GEX_INIT sent
debug1: expecting SSH2_MSG_KEX_DH_GEX_REPLY
debug3: check_host_in_hostfile: host 127.0.0.1 filename /.ssh/known_hosts
debug3: check_host_in_hostfile: host 127.0.0.1 filename /.ssh/known_hosts
debug3: check_host_in_hostfile: host 127.0.0.1 filename /opt/ssh/etc/ssh_known_hosts
debug3: check_host_in_hostfile: host 127.0.0.1 filename /opt/ssh/etc/ssh_known_hosts
debug3: check_host_in_hostfile: host 127.0.0.1 filename /.ssh/known_hosts
debug3: check_host_in_hostfile: host 127.0.0.1 filename /opt/ssh/etc/ssh_known_hosts
debug2: no key of type 0 for host 127.0.0.1
debug3: check_host_in_hostfile: host 127.0.0.1 filename /.ssh/known_hosts2
debug3: check_host_in_hostfile: host 127.0.0.1 filename /opt/ssh/etc/ssh_known_hosts2
debug3: check_host_in_hostfile: host 127.0.0.1 filename /.ssh/known_hosts
debug3: check_host_in_hostfile: host 127.0.0.1 filename /opt/ssh/etc/ssh_known_hosts
debug2: no key of type 2 for host 127.0.0.1
The authenticity of host '127.0.0.1 (127.0.0.1)' can't be established.
RSA key fingerprint is 9f:57:2f:04:b9:ec:bd:91:45:2e:d9:29:38:ff:ee:c5.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '127.0.0.1' (RSA) to the list of known hosts.
debug2: bits set: 523/1024
debug1: ssh_rsa_verify: signature correct
debug2: kex_derive_keys
debug2: set_newkeys: mode 1
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug2: set_newkeys: mode 0
debug1: SSH2_MSG_NEWKEYS received
debug1: Roaming not allowed by server
debug1: SSH2_MSG_SERVICE_REQUEST sent
debug2: service_accept: ssh-userauth
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug2: key: /.ssh/identity (0)
debug2: key: /.ssh/id_rsa (0)
debug2: key: /.ssh/id_dsa (0)
debug1: Authentications that can continue: publickey,password,keyboard-interactive
debug3: start over, passed a different list publickey,password,keyboard-interactive
debug3: preferred publickey,keyboard-interactive,password
debug3: authmethod_lookup publickey
debug3: remaining preferred: keyboard-interactive,password
debug3: authmethod_is_enabled publickey
debug1: Next authentication method: publickey
debug1: Trying private key: /.ssh/identity
debug3: no such identity: /.ssh/identity
debug1: Trying private key: /.ssh/id_rsa
debug3: no such identity: /.ssh/id_rsa
debug1: Trying private key: /.ssh/id_dsa
debug3: no such identity: /.ssh/id_dsa
debug2: we did not send a packet, disable method
debug3: authmethod_lookup keyboard-interactive
debug3: remaining preferred: password
debug3: authmethod_is_enabled keyboard-interactive
debug1: Next authentication method: keyboard-interactive
debug2: userauth_kbdint
debug2: we sent a keyboard-interactive packet, wait for reply
debug2: input_userauth_info_req
debug2: input_userauth_info_req: num_prompts 1
Password:
debug3: packet_send2: adding 32 (len 22 padlen 10 extra_pad 64)
debug2: input_userauth_info_req
debug2: input_userauth_info_req: num_prompts 0
debug3: packet_send2: adding 48 (len 10 padlen 6 extra_pad 64)
debug1: Authentication succeeded (keyboard-interactive).
Authenticated to 127.0.0.1 ([127.0.0.1]:22).
debug1: Final hpn_buffer_size = 131072
debug1: HPN Disabled: 1, HPN Buffer Size: 131072
debug1: channel 0: new [client-session]
debug3: ssh_session2_open: channel_new: 0
debug2: channel 0: send open
debug1: Requesting no-more-sessions@openssh.com
debug1: Entering interactive session.
debug2: callback start
debug2: client_session2_setup: id 0
debug2: channel 0: request pty-req confirm 1
debug2: channel 0: request shell confirm 1
debug2: fd 4 setting TCP_NODELAY
debug2: callback done
debug2: channel 0: open confirm rwindow 0 rmax 32768
debug2: channel_input_status_confirm: type 99 id 0
debug2: PTY allocation request accepted on channel 0
debug2: channel 0: rcvd adjust 65536
debug2: channel_input_status_confirm: type 99 id 0
debug2: shell request accepted on channel 0
#############################################################################
#############################################################################

debug2: channel 0: request pty-req confirm 1
debug2: channel 0: request shell confirm 1
debug2: fd 4 setting TCP_NODELAY
debug2: callback done
debug2: channel 0: open confirm rwindow 0 rmax 32768
debug2: channel_input_status_confirm: type 99 id 0
debug2: PTY allocation request accepted on channel 0
debug2: channel 0: rcvd adjust 65536
debug2: channel_input_status_confirm: type 99 id 0
debug2: shell request accepted on channel 0
Last successful login: Fri Sep 28 14:52:23 METDST 2012 10.4.3.97
Last authentication failure: Fri Sep 28 14:52:16 METDST 2012 10.4.3.97
Last login: Fri Sep 28 14:52:43 2012 from 10.4.3.97
(c)Copyright 1983-2006 Hewlett-Packard Development Company, L.P.
(c)Copyright 1979, 1980, 1983, 1985-1993 The Regents of the Univ. of California
(c)Copyright 1980, 1984, 1986 Novell, Inc.
(c)Copyright 1986-2000 Sun Microsystems, Inc.
(c)Copyright 1985, 1986, 1988 Massachusetts Institute of Technology
(c)Copyright 1989-1993 The Open Software Foundation, Inc.
(c)Copyright 1990 Motorola, Inc.
(c)Copyright 1990, 1991, 1992 Cornell University
(c)Copyright 1989-1991 The University of Maryland
(c)Copyright 1988 Carnegie Mellon University
(c)Copyright 1991-2006 Mentat Inc.
(c)Copyright 1996 Morning Star Technologies, Inc.
(c)Copyright 1996 Progressive Systems, Inc.

Confidential computer software. Valid license from HP required for
possession, use or copying. Consistent with FAR 12.211 and 12.212,
Commercial Computer Software, Computer Software Documentation, and
Technical Data for Commercial Items are licensed to the U.S. Government
under vendor's standard commercial license.

You have mail.

Value of TERM has been set to "xterm".
WARNING: YOU ARE SUPERUSER !!

Thanks a lot

Have you checked for high load or low available free RAM on the server?

Hi,

This isssue is on all the HP-UX servers, and there is no Ram or high load issue.

Thanks for answer,

When you say "all the HP-UX" server you mean others (AIX, SOLARIS...) have no?

We've got Red Hat Linux RHEL5 but at this box the SSH is running well.

At HP-UX boxes is where the SSH connection is slow. Also we 've realised that doing a telnet connection throw HP-UX boxes is working fine, it's seems to be so the main issue is that it'll be a configuration parameter that doing slow connection.

Cheers.

Hi, it maybe could be an issue with the dns resolver.
If the dns entries are not resolvable, it could take longer to connect with ssh to your server.

Could you elaborate on the term "slow connections".
Is the etablishion of a connection slow, or did you get slow transfer rates from ... to ssh.

cheers ...

---------- Post updated at 02:04 PM ---------- Previous update was at 01:49 PM ----------

check /etc/resolv.conf
Did you have a system without such this problem, then compare
Is the IP your are trying to connect available in /etc/hosts
Check your etc/nsswitch.conf file
hosts: files dns
ipnodes: files dns
Should have entries in above orders.
Are there problems with your main dns server in case that the second dns server answers your dns queries?
What says the /var/adm/syslog/syslog.log about your problem . Check it!

Hi,

We have set only nsswitch.conf in some systems HP-UX:

# cat etc/nsswitch.conf
hosts: files [NOTFOUND=continue UNAVAIL=continue TRYAGAIN=continue] dns
 
# cat /etc/resolv.conf
cat: Cannot open /etc/resolv.conf: No such file or directory

In this case, the connection is slow.

In other cases, we have set the two files:

# cat etc/nsswitch.conf
hosts: files [NOTFOUND=continue UNAVAIL=continue TRYAGAIN=continue] dns
 
# cat resolv.conf
search LLL.com
nameserver XXX
nameserver YYY

In this case, the conection is slow, too.

I checked in the syslog.log i cannot see error related.

Thanks a lot,

Noting these lines from your log of your ssh session:

 
Last successful login: Fri Sep 28 14:52:23 METDST 2012 10.4.3.97
Last authentication failure: Fri Sep 28 14:52:16 METDST 2012 10.4.3.97
Last login: Fri Sep 28 14:52:43 2012 from 10.4.3.97
 

When I run a log on my HP-UX 11.31 box with:

 
ssh -v -v -v root@127.0.0.1 

the lines for me give me hosts names, not IP addresses.

This means the login process is doing a revese lookup on the IP address to provide a host name, and that requires it either have the entry in /etc/hosts, or the hosts be correctly set up to use dns.

For your servers that do not have an /etc/resolv.conf file, you are not set up correctly to use dns and this means that the delay is most likely caused by the reverse lookup timing out, because the host can't find a dns server to use (how could it? It's not configured.)

For the systems that do have an /etc/resolv.conf file, you may want to check that the XXX and YYY servers are actually dns servers, and you can ping them from the current host in question, and that you are searching on the correct network domain(s).

As a simple test ... on the system that does not have an /etc/resolv.conf file set up,
can you change the /etc/nsswitch.conf file hosts entry to:

hosts: files

removing the rest of the line? In this way you tell the system not to look for a dns server if you can't find the entry in /etc/hosts, and therefore it will decide to return the IP Address on the reverse lookup more quickly.