Solaris 10 /proc making filesystem full

Hi all,

I have a solaris 10 box on which /proc is a part of /
The problem is that my / partition is getting full mainly
due to /proc getting a lot of files.

My question is : Can I delete files/directories in /proc directory.
If not what could be the other way round to clean up /proc so that I can
free some of the resources.

Thanks,
rte

/proc is not part of /. /proc does not contain real files on disk anywhere. Post a df -k of your filesystems.

Here is the output of my df -k command

bash-3.00# df -k
Filesystem            kbytes    used   avail capacity  Mounted on
/dev/dsk/c0t0d0s0    24203976 21756436 2205501    91%    /
/devices                   0       0       0     0%    /devices
ctfs                       0       0       0     0%    /system/contract
proc                       0       0       0     0%    /proc
mnttab                     0       0       0     0%    /etc/mnttab
swap                 16648996     732 16648264     1%    /etc/svc/volatile
objfs                      0       0       0     0%    /system/object
/dev/dsk/c0t0d0s1    16137782 4103655 11872750    26%    /usr
/usr/lib/libc/libc_hwcap2.so.1
                     16137782 4103655 11872750    26%    /lib/libc.so.1
fd                         0       0       0     0%    /dev/fd
/dev/dsk/c0t0d0s4    10085921   83584 9901478     1%    /var
/dev/dsk/c0t0d0s5    3891708    7020 3845771     1%    /tmp
swap                 16648312      48 16648264     1%    /var/run
/dev/dsk/c0t2d0s0    51642279 5544335 45581522    11%    /zone_1
/dev/dsk/c0t3d0s0    51642279 4492545 46633312     9%    /zone_2
/dev/dsk/c0t1d0s0    51642279 12978094 38147763    26%    /zone_3

Thanks
rte

Upon closer inspection of /proc it contains directories in the name of running PIDs. In my case Apache which has about 35 processes running (shown in ps -ef). These directories are large (some of them are about 700MB or less).

I wonder if these instances of Apache are making my / filesystem full.

Any clues?

Thanks,
rte

Stop looking at /proc as your issue - as Perderabo stated,

Take a look at filesystem full what to look for. Don't try anything that you are not SURE of. Removing files from a system is not something you want to do unless you know what you are doing/deleting.

Look for core or tar files that may have been put on by another person. Since /var and /usr are separate filesystems, don't bother checking them. Check /etc (be CAREFUL!!!) and /opt.

Look at the final field on your df listing. Those are your separate filesystems. Stuff in /proc, /usr, /var, /zone_1, and so on is not in /. And stuff in /proc is nowhere except in memory. As you say, it is your pids. You can delete stuff in /proc by killing processes, but this will not help you. /proc is not a real filesystem, it is a way to access your processes as if they were files.

/ should not grow very much. Has it grown in, say, the past 10 days? Then get a list of files in / modified in the past 10 days...
find / -mount -type f -mtime -10
This will probably be a short list and it should tell you where the problem is.

Just wondered, if there is no real files in /proc, why does it show up as the highest space occupier on my server (this is an old Solaris 2.6):

#du -sk * |sort -rn |head
723555 proc
302662 usr
282955 opt
249259 export
209532 var
8568 tmp
5997 kernel
5019 sbin
3965 platform
2536 etc

-----Post Update-----

I would appreciate a quick response, Thanks.

The /proc system in Solaris uses procfs. Think of it as any other vfs, but instead of representing data on disk, it represents the process structure of running applications. The reason you are seeing it as taking up space is because procfs contains a representation of a given processes address space, as, and as such can be quite large.

Also, keep in mind the command you issued traverses mounted filesystems.

Finally, as the title suggested, "everything is a file." This is a fundamental Unix design principle. It is why devices, ports, virtual terminals, etc, are all accessed as files and through standard system calls like open(), for example. /proc is just extending the "everything is a file" principle to include processes for ease of access. It makes creating (and using) tools like 'pgrep', 'pkill', 'preap' (a personal fav), 'pmap', etc., easy.

Since /export is part of / on your system and home directories are usually in /export/home on Solaris you might want to check the home directories of your users for new files that are taking up too much space. The same can be said for /opt where some applications may be storing things like log files.

Please leave /proc alone. :slight_smile:

Thank you for the explanation forsnick.

Seg, du command that I run does not care wether directories are on separate partitions or not. On this server, /usr, /opt, /export/home, and /var are all on their own partitions so removing things from them would not solve my problem.

I have found stuff to remove for now until we rebuild this old server with an appropraite size /.
Thanks all.

Sara-sh,
I was responding to the original poster. I see now that you resurrected an old thread to bring up your point- my reply was not intended to match your situation.

Sorry about the confussion Seg ...

You guys are great, thanks for sharing the knowledge, appreciate it.

I have been face it myself at my machine. I just reboot the OS-solaris then /proc going to normal again.

Regards

Bejo :slight_smile:

Thanks Bejo, that is a good point. However the quetion is, was there any disk space released as a result of the reboot, was there more space in / directory after the reboot?
Thanks again ...

Um, rebooting and making "/proc going to normal again" is flawed reasoning and not sound advice technically. /proc appears to grow larger as your machine runs because it contains a representation of running applications memory (NOT DISK) space. So, rebooting makes /proc smaller because you're no longer running those applications. As soon as you start running applications again, /proc will grow as memory gets addressed. This is normal behavior!

>>> /proc has nothing whatsoever to do with disk space. <<<

Side note: don't reboot Unix machines arbitrarily, it makes Unix admins sad.

Thanks again

After you reboot your machine /proc capacity wil be 0%.

Best Regards

Bejo:)

I am having the similar problem too. proc and dev have taken up huge amount of the space. Will a reboot clear dev too? Currently /home2, 3 and 4 are on different slices.

0	du.txt
0	home
0	net
0	vol
1	bin
1	cdrom
1	export
1	home10
1	home11
1	mnt
8	lost+found
9	home1
12	INFORMIXTMP
26	hdir
88	tmp
112	axinstall26
147	devices
207	boot
1511	sbin
4838	InstallShield
13565	system
31314	lib
31823	test0515
41856	kernel
63447	etc
255356	platform
2033773	opt
2048006	var
3366136	usr
4180789	opt2
12337612	home4
12798104	home2
17410368	home3
17452416	dev
20503084	proc

I guess that I will try one more time... /proc is an illusion. It does not actually consume disk space.

If you create a new process with, say pid 1234 and it is using 300 MB of memory, /proc/1234 will magicly pop into existence and it will be 300 MB in size. But you did not lose any disk space. Not one byte.

Now, kill pid 1234 and /proc/1234 will vanish. But you won't get back even one byte of disk space. /proc/1234 is just a way to treat a process as if it was a data file. This makes stuff like debuggers very easy to write.

/dev, on the other hand, is probably a real problem. It is usually caused by someone doing something like:
tar cvf /dev/wrong-name-for-tape-drive /big/collection/of/files

Maybe:
find /dev -type f
will find it for you.

Hola,
i have found the culprit leading to high root usage. /dev/rmt/0n has taken huge amount of disk space.

Things have went back to normal now! Thanks for your 'clue'. =)