We run an RS/6000 SP Frame. One of the nodes running (AIX 5), in the frame, seems to run fine for a few weeks and then will no longer communicate with the outside world. I can not telnet to the server or ping the server.
This is true for both the ethernet and serial connection.
I believe what is happening is that there is a process running out of control, eating all the system resources and then finally bringing the system to its knees.
We are running Oracle 9i on the system as well.
Does anyone have any ideas how to monitor the system before it crashes.
I have looked into the syslogs and dumps but it doesn't seem to contain any information about why the system dies.
If it is not a production server, try to stop the Oracle service and see the result. May be it is caused by the data load or misconfiguration between AIX and the Oracle.
Have you check the errlog. May be this would help. Get the error code and check on IBM site.
Monitoring the log is tedious job. In addition, monitoring the log before it's crash is not a solution to your problem.
Thanks for the reply alisetan.
Unfortunatley the server is a production server and I will not be able to bring down the databases and stop the services.
I have run an errpt -a but the last entry before the server died was from 10 days ago and the next entry is from the system starting up.
I think at this point I am going to run ps on a 10 minute schedule and grep all oracle instances, and pipe it out to a file.