Hi all,
One of our programs written in Java, produced this logfile. This job runs 48 threads and only one thread failed with this error. The code is a blackbox(external product), so cant look at the source code. From what I can infer from the log, the job was trying to write the log messages into a file but didnt write and failed. THe error is below
There was no core dump too. We were having swap space issues on this box previously. our admins increased the swap 2 days back. The next day, the job ran fine. But the subsequent run failed and the error is above. Can anyone throw some light on this.
Box: HP-UX B.11.11
swapinfo -t gives
I ran it seperately. But once, while monitoring the job, we got a Red alert in 'glance' saying that the 'Global swap space is nearly full'. It was after pointing out this that the swap was increased.
No signal in syslog. Is there any command to see which job is taking up more virtual memory? If we can know the application using the resources, we can atleast ask for a rescheduling of the job.
The lsp_engine - looks like a virtual machine - is the one with the problem. If the virtual memory problem started with running this app, you don't need to worry about who/what is using memory. Try to run the app as close to solo as possible.
I have not done sysadmin for long time - maybe somebody like Perderabo can give you tuning help. What I see is that you have a lot of free pagefile space.
It may also be that you will have to reconfigure the app, if that is possible.
The job is running fine for the past 2 days. We raised the issue with our Sysdamins and they havent got back to us. It looks like they are running 2 similar applications(a new project in Acceptance box) and both have multiple threads running at the same time. Our DBA's advise was that after the testing one App would be switched off and this issue shouldnt occur.
So, what we have determined to do in case of job failure is to check the system status at that moment and the jobs currently running and try to rerun the job after some time when the activity on the system has decreased.
I hope that will help.
Thanks for your help on this. I will get back if we find anything else.
This issue arose again. The job abended today with the same issue. I requested a rerun for the job. It runs 48 threads. After 4 minutes, one of the threads aborted with the error mentioned in this thread. All the other threads ran fine and completed. Below are the snapshots of the commands run while the job was executing. I have put the details in the attachment. Hope it throws some light.