high cpu utilization

UNIX for Advanced & Expert Users
1

good morning.

just wanted to ask if there's a way to check what causes the high cpu utilization of a server for the past 2 months? My jffnms report resulted to high utilization for a specific server last month. is there a way to check via a command line?

thanks

2

You might have to observe or log a while to see. I have used a simple script to find processes that are using high CPU, where I do a ps -ef every N seconds, and compare them, suppressing lines with 0-1 second CPU used, using some sort, comm -13 and grep. Logging time for each pass is good, too, so you can see who runs what when to drive it up. You may find a looping process right off the bat. It goes something like this:

#!/usr/bin/ksh

export fa=/tmp/ps.a.$$ fb=/tmp/ps.b.$$

ps -fp$$ | read zh  # capture ps header

ps -ef | sort >$fb   # prime the comparison

while [ 1 ]
do
 sleep 120
 mv $fb $fa
 date
 ps -ef | sort >$fb
 echo "$zh"
 comm -13 $fa $fb | grep -v '0:0[01] '
 echo
done
3

thanks..does this means that there is no way to check the past month say last december? pls advise

4

Well, if the system logged tremendous detail unasked, it would be a dog. You can find how many times every command was called, but I don't think it tracks total cpu cycles. There may be accounting bits to tell you user cpu time totals, but they may have to be turned on. Usually, it is something to do ongoing forever, so you can tell if it is a command or a user or batch or service or possible attack/denial of service.

Using all the CPU you bought is no sin. (There is a psychology of reserve that says it is, but it is just neurosis. Once, a manager suggested I not fix the CPU with the last spare PWB, because then we would not have a spare . . . .) If the users feel it is slow, then look.

5

thank you for your reply.
just a bit confused.

Tried the below and here's one output of the script your wrote.

     UID   PID  PPID   C    STIME TTY         TIME CMD
    root     0     0   0   Dec 03 ?           3:02 sched
    root     1     0   0   Dec 03 ?           0:02 /sbin/init
    root     2     0   0   Dec 03 ?           0:00 pageout
    root     3     0   0   Dec 03 ?         105:39 fsflush
    root     7     1   0   Dec 03 ?           0:12 /lib/svc/bin/svc.startd
    root     9     1   0   Dec 03 ?           0:32 /lib/svc/bin/svc.configd
    root  3985     1   0   Dec 14 ?          32:25 /usr/jdk/instances/jdk1.5.0/bin/java 
   smmsp   910     1   0   Dec 03 ?           0:02 /usr/lib/sendmail -Ac -q15m
  daemon   163     1   0   Dec 03 ?           0:00 /usr/lib/crypto/kcfd
  daemon   311     1   0   Dec 03 ?           0:00 /usr/sbin/rpcbind
-Dcom.sun.aas.instanceRoot=/opt/SUNWappser
 lhareigh 10962 10961   0 14:32:25 ?           0:00 /usr/lib/ssh/sshd

a. how will I know which CPU are they occupying? see output of psrinfo

The physical processor has 8 virtual processors (0 2 4 6 8 10 12 14)
  x86 (chipid 0x0 GenuineIntel family 6 model 26 step 5 clock 2933 MHz)
        Intel(r) Xeon(r) CPU           X5570  @ 2.93GHz
The physical processor has 8 virtual processors (1 3 5 7 9 11 13 15)
  x86 (chipid 0x1 GenuineIntel family 6 model 26 step 5 clock 2933 MHz)
        Intel(r) Xeon(r) CPU           X5570  @ 2.93GHz

b. on the output above, which of them has the highest CPU usage and what CPU it uses?

thank you for your swift reply

6

If you have the "top" command it is very good for showing you the CPU usage live. Most versions also show you which CPU a process is using. Very useful for finding orphan looping processes.

What Operating System and version are you running?

Apart from unix process accounting (an acquired taste) there are many commercial monitoring suites of programs available. Very detailed record keeping is a significant task and may cause you to re-size your server in order to maintain the historic records.
Setting up basic unix System Activity recording (see "man sar" and "man sadc") is useful if you want an overview.

Btw. High average CPU usage often just means that you have sized your server correctly. If processes are constantly queueing for CPU time, then you have a problem.
Beware that some performance monitors know how many processors you have and can report say 800% utilisation (on a 16 processor system) without there being a problem. In this example 100% utilisation is trivial.

7

The CPU in use by a process may vary for every run time slice, although for some situations it helps to run the same lwp on the same CPU if possible.

Maybe with some wide ps options (w or x, it varies), you can see what the java proc is running. You can make the interval lower, and see who shows up often. Sometimes the scheduling column changes, and you need a ps -o (as I recall) with specific columns relevant to who is using how much of what.

Top is great, too, but can be a bit busy and leaves no log.

8

It seems it is Solaris 10.

In Solaris 10, "svc:/system/sar" create cron job "/var/spool/cron/crontabs/sys" to record sar output to "/var/adm/sa"

The sar ouput history can be viewed by "-f".

$ sar -u -f /var/adm/sa/sa24

00:00:01    %usr    %sys    %wio   %idle
01:00:01       0       0       0     100
02:00:00       1       1       0      98

Linux with "sysstat" pkg installed will output to /var/log/sa,

9

Some times top is not working solaris old versions. better go with sar(system activity report) command.

10

If this is Solaris, see "man sadc" for sample crons for data collection for "sar".