Hi everybody,
I have an Unix box running Solaris and every day for 1 hour or 2 the box is stuck and I can only get this error message when trying to type a command :
bash-3.00$ vmstat 5
bash: fork: Resource temporarily unavailable
How can I trace what's is going wrong with this box ? Which commands are useful to run before and after the issue ?
Thanks a lot !
Fabien
Sounds like you are running out of swap, try putting a cronjob in to run "df" very 5 minutes and append to a file, then after the next time it happens go back and see if usage of /tmp was growing.
Alternatively, do the same interactively and see if someone is writing to a file or files in /tmp and filling it.
If you can't fork you can't run a command in a shell to see why it can't fork, as it needs to fork to create the process to run the command in.
First, reborg suggested arranging for a command to run every five minutes which might show /tmp filling prior to being totally filled. And it is possible for a large process to be unable to fork while smaller processes have no problem. This is actually quite common. Finally, it is not exactly true that a shell must fork to run a command. A shell is programming language with lots of built in commands. Can't launch ls?
$ myls() { while [ $# -ne 0 ] ; do echo "$1" ; shift ; done ; }
$ myls /etc/s*
/etc/services
/etc/shells
/etc/syslog.conf
$
Can't launch cat?
$ mycat() { while IFS="" read l ; do echo "$l" ; done < $1 ; }
$ mycat /etc/shells
#
# This is a list of valid login shells (single rooted).
# Ftpd must see one of these shells as a user's login shell to connect.
#
/bin/sh
/bin/ksh
/bin/csh
/bin/tcsh
/usr/local/bin/bash
$
None of that needed a single fork provided I use ksh. I can conjure up limited versions of quite a few commands this way. And if I am desperate to diagnose a stalled OS before I cycle power, I have a final trick...I can run a single command by using exec...
exec df -k
A box that cannot fork can often exec.
Quite right of course and as an extension, some bigger process might die because it runs out of memory, leaving enough room for applications with a smaller memory footprint to run quite happily.
Both of the suggestion I posted were to run prior to the problem happening. The cron to give a general idea of what was happening, or more simply to periodically run the commands before the problem happened which might in fact point out the exact cause before the problem happened for example some application or script writing a huge log file to /tmp.
Perderabo,
You have any other functions in that neat bag of tricks? Those functions are really helpful.
-S