Unable to ssh or login to AIX server

UNIX AIX
1

We are having occasional problems accessing some AIX servers. When this happens we cannot ssh to the server in question or login via HMC console terminal window. We can ssh some commands to the server and get responses but other commands just hang, ssh serverA date returns the date, ssh serverA df -g hangs. This has happened on as least 3 different VLPARS all at AIX 7100-02-03-1334. Any thoughts or suggestions would be welcome, we also have an active PMR open with IBM support but nothing concrete from that yet.

Thanks Kierong

2

You should login and run the df -g command interactively to insure the command itself is not hanging

 ssh date

versus

ssh df -g

have different performance impact on a server.

3

What applications are hosted by those LPARS?
I'd set up sar or nmon to monitor them to see what is happening during the time you can not log in.

4

It may not be necessarily the problem of network. When this happens or after this happen, make sure you check

  1. Memory utilization
  2. Disk - File system utilization on (/) root and /usr file systems.
  3. CPU utilization.
5

Memory looks good no paging taking place. Was able to ssh the lsps -a command and get a response 2% utilized. nmon and sar show CPU usage normal no spikes these utilities are always gathering data.. Filesystem usage appears normal but we cannot tell as we cannot log into the server to check. If we ssh a df -k command it never comes back. No problem errors reported in the error report.

6

There could be something that uses pinned memory and the space allocation may have gone wrong. This is memory that cannot be swapped out, such as a database may use.

Another alternate we had was when we applied a TL to a server. It installed a number of things and forced up the pinned memory required that way. Somewhere I've got a list of what we switched off. The LPAR only has 1Gb RAM in any case.

Robin