I have RHES4 machine with VRTSralus - Backup Exec agent installed there and running as a service. The agent hiccups sometimes and turns into defunct state. The problem is that I cannot kill it anyway., it stays there forever until the machine is rebooted. I wonder if anyone had such an experience and know the way to kill zombie service.
Here is the line from `ps aux` under normal condition:
root 2593 0.0 0.0 66612 7064 pts/1 S 16:13 0:00 /opt/VRTSralus/bin/beremote
The next one is I mentioned before which I cannot kill unless I restart the server whci is not an option as it is a production Oracle DB server.
root 10183 2.5 0.0 0 0 ? Z Aug25 370:58 [beremote <defunct>]
I guess it is a zombie one:
[root@jdedb2 root]# ps -elf | grep Z
F S UID PID PPID C PRI NI ADDR SZ WCHAN STIME TTY TIME CMD
4 Z root 10183 1 2 75 0 - 0 t> Aug25 ? 06:10:58 [beremote <defunct>]
4 Z root 27536 1 2 75 0 - 0 t> Sep02 pts/1 01:11:55 [beremote <defunct>]
I need to add that on the top of these zombies there is unusial iowait %. I do not know if it is related to these zombies, but I wonder if anybody knows how to trace what process is causing high iowait %
[root@jdedb2 root]# top
17:48:39 up 19 days, 18:54, 3 users, load average: 3.24, 3.14, 3.17
392 processes: 389 sleeping, 1 running, 2 zombie, 0 stopped
CPU states: cpu user nice system irq softirq iowait idle
total 3.2% 0.0% 3.6% 0.0% 0.0% 200.4% 192.0%
cpu00 0.0% 0.0% 0.0% 0.0% 0.0% 100.0% 0.0%
cpu01 1.9% 0.0% 2.9% 0.0% 0.0% 2.9% 92.0%
cpu02 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 100.0%
cpu03 1.5% 0.0% 0.7% 0.0% 0.0% 97.6% 0.0%
Mem: 16304516k av, 16194096k used, 110420k free, 0k shrd, 55232k buff
11648432k actv, 2245884k in_d, 324248k in_c
Swap: 12287992k av, 0k used, 12287992k free 13305320k cached
I appreciate if you can help.