prstat log

UNIX Solaris
1

Hi All,

But the prstat logs of my system shows:-
NPROC USERNAME SWAP RSS MEMORY TIME CPU
77 oracle 17G 10G 65% 4:24:47 0.8%
Total: 486 processes, 3850 lwps, load averages: 3.77, 4.45, 4.94

What does the MEMORY denotes?
Is it the %memory used from RAM?
Or is the %memory used by user it has been allocated?
What does these parameters denote:-
NPROC USERNAME SWAP RSS MEMORY TIME CPU?

Is the %memory usage of oracle user in the prstat log normal?
Should it affect the system and apparently cause system to crash?

Please confirm whether the oracle user can use more memory than the SGA+PGA taken together.

Thanks in advance.

2

Nope its not normal. check your oracle instances and kill those which are hogging. I believe you should know what to do.
Increase the swap space in your system.

3

It just sums the shared memory for every process and this is obviously wrong.
You should ignore that report.

4

The SGA is allocated per oracle instance. The prstat total statistics are wrong as far as the shared memory is concerned.
The PGA is only a target (soft limit) so yes, an oracle process can use more than that limit. Note that there are other memory areas (outside of the PGA) to consider too.

5

Hey Thanks...

Iam new to this. Can you tell me what will be the impact of it? And this % memory usage was there in the lab for more than 3 days. Its back to 8.5% now. when there was an increase in memory consumption the system crashed but in 3 mins it was back to normal. But still the %memory was as high as 62%.

At the time of fault Following is my investigation on the prstat logs for the three days:-

  1. At Sat Oct 11 20:22:01 CST 2008 (When the problem did not occurred)
    NPROC USERNAME SWAP RSS MEMORY TIME CPU
    74 oracle 3580M 3571M 22% 4:16:30 12%

Please note there is no record of the prstat results in the log file till Sat Oct 11 20:29:11 CST 2008.

  1. At Sat Oct 11 20:30:05 CST 2008
    NPROC USERNAME SWAP RSS MEMORY TIME CPU
    74 oracle 17G 10G 61% 4:18:36 1.2%
    Total: 508 processes, 4305 lwps, load averages: 37.23, 56.34, 30.41
  2. At Sat Oct 11 20:33:01 CST 2008(When the problem got resolved)
    NPROC USERNAME SWAP RSS MEMORY TIME CPU
    75 oracle 17G 9905M 60% 4:19:00 1.6%
    Total: 499 processes, 4303 lwps, load averages: 10.29, 36.75, 27.04
  3. At Sat Oct 11 21:25:00 CST 2008(After the problem got resolved the memory was still high)
    NPROC USERNAME SWAP RSS MEMORY TIME CPU
    77 oracle 17G 10G 64% 4:22:38 3.4%
    Total: 496 processes, 3982 lwps, load averages: 4.47, 4.82, 5.86

Generally the %memory usage of Oracle during sane state varies within 8.4% to 9%.
What could be the probable reason for the discrepancies in the memory usage by oracle during the time of fault and during sane condition?

Thanks.

6

Well,
could you explain what you mean by system crash? It seams that there was an oracle instance crash. I would examine the instance alert log first. I think that the prstat log won't help in this case.

7

Hi ,

With system crash I meant that the GUI seesions launched terminated and telnet was not possible and no operations could be performed at the system.

The alert log is the following:-
Sat Oct 11 20:24:10 2008
Process startup failed, error stack:
Sat Oct 11 20:24:11 2008
Errors in file /opt/oracle/u01/admin/CEMSDB/bdump/cemsdb_psp0_28290.trc:
ORA-27300: OS system dependent operation:fork failed with status: 11
ORA-27301: OS failure message: Resource temporarily unavailable
ORA-27302: failure occurred at: skgpspawn3
Sat Oct 11 20:24:12 2008
Process q002 died, see its trace file
Sat Oct 11 20:24:12 2008
ksvcreate: Process(q002) creation failed
Sat Oct 11 20:24:48 2008
Process startup failed, error stack:
Sat Oct 11 20:24:48 2008
Errors in file /opt/oracle/u01/admin/CEMSDB/bdump/cemsdb_psp0_28290.trc:
ORA-27300: OS system dependent operation:fork failed with status: 11
ORA-27301: OS failure message: Resource temporarily unavailable
ORA-27302: failure occurred at: skgpspawn3
Sat Oct 11 20:24:49 2008
Process q002 died, see its trace file
Sat Oct 11 20:24:49 2008
ksvcreate: Process(q002) creation failed
I had checked specified cemsdb_psp0_28290.trc trace file but didn't find more details.

8

Which Solaris version?
Could provide also the output from the following command (run as oracle):

ulimit -n
9

Hi
$ ulimit -n
2048

$ uname -a
SunOS <> 5.10 Generic_127127-11 sun4u sparc SUNW,Sun-Fire-V240

10

Hi, is it possible that your filesystem ist running full? You got oracle under /opt.

If you need a better human readable format for these ouputs try sysstat - there are no binaries, you have to compile it.

11

I saw you've already posted the problem on the Oracle forums but I think you should rather open an SR to Oracle Support. Status 11 seems to be related to number of processes OS limits (it should be 12 for memory shortage).
Hence I would try to increase process.max-file-descriptor and try to reproduce the problem.