/tmp filesystem full

I am running AIX 3 4.

When I do a df I get:

Filesystem    512-blocks      Free %Used    Iused %Iused Mounted on
/dev/hd4           32768     10232    69%     1309    16% /
/dev/hd2          917504     86360    91%    19744    18% /usr
/dev/hd9var       131072     67712   49%      617     4% /var
/dev/hd3           65536         0      100%       72     1% /tmp
/dev/lv00         196608     35064     83%      899     4% /usr/welcome
/dev/lv02          32768     20568      38%      580     4% /usr/ob
/dev/lv01        1441792    656008    55%     4127     1% /home
/dev/lv04         425984    294376     31%     1319     1% /apps
/dev/lv03        1605632    238520    86%     1187     1% /u1
/dev/lv05         425984    272424     37%     9315     9% /u2
/dev/lv06        2031616   1188840    42%      751     1% /wk

This shows the /tmp is 100% used. I have looked in every /tmp directory I can find (by using 'find') and there are no big files filling up the space. How do I find the files that are filling up the space? I have already deleted all the 'core' and smit.log files. This happened a week ago and I cannot free up the space. Can anyone please help?

Thanks!

Steve

cd /tmp
du -sk * | sort -nr | more

This will give you a full listing of directories/files in /tmp, in the order of the space they occupy. Then you can check out the individual files/directories and purge.

The only files I can find of any size are in the /tmp/vgdata/rootvg:

# cd vgdata
# ls
rootvg
# ls -l rootvg
total 112
-rw-r--r--   1 root     sys          336 Jul 08 23:32 hd2.map
-rw-r--r--   1 root     sys           24 Jul 08 23:32 hd3.map
-rw-r--r--   1 root     sys           12 Jul 08 23:32 hd4.map
-rw-r--r--   1 root     sys           12 Jul 08 23:31 hd5.map
-rw-r--r--   1 root     sys          144 Jul 08 23:31 hd6.map
-rw-r--r--   1 root     sys           12 Jul 08 23:31 hd8.map
-rw-r--r--   1 root     sys           48 Jul 08 23:32 hd9var.map
-rw-r--r--   1 root     sys           72 Jul 08 23:32 lv00.map
-rw-r--r--   1 root     sys          528 Jul 08 23:32 lv01.map
-rw-r--r--   1 root     sys           12 Jul 08 23:32 lv02.map
-rw-r--r--   1 root     sys          588 Jul 08 23:32 lv03.map
-rw-r--r--   1 root     sys          156 Jul 08 23:32 lv04.map
-rw-r--r--   1 root     sys          156 Jul 08 23:33 lv05.map
-rw-r--r--   1 root     sys          744 Jul 08 23:33 lv06.map

I'm not familier with .map files - can I safely delete these?

Steve

You need to use a program like fuser or lsof on /tmp. Files do not always have names. If a process has a file open, the space will not be released just because you unlink the only file name it has.

Yes, it is possible that a process is stuck in a loop and writing continuously to a log or a debug file. Can you check 'top' and see if any process is taking up an unusual amount of CPU %age?

There doesn't seem to be a 'top' command on this system.

Steve

Well, use whatever tools AIX has to see processes. If this has been going on for a week, the process will be that old. Can you reboot? That will free the space.

Good question! We are concerned that if we try to reboot the system it will hang due to the /tmp being full. Is that a valid concern?'

Steve

I don't use AIX, but I can't imagine why it would. I would reboot.

this could be one of the possible case...

check whether any of ur process is using IPC's like shared memory, mesage queues etc

and check out where these segement ID's are given in .

If so in swap space /tmp that could be a problem

This is just one possible reason.

In solaris, I will do a prstat -avm, in linus, I will do a top, in AIX, i dunno.
I will look for the largest resident size (memory used).

E.g. Try sending out a large mail from unix, you will see the /tmp is used when the physical memory is used up. The resident memory will increase steadily at the same time under the sendmail process.
Kill the sendmail process, /tmp will be back to 0%.

In AIX you could use "topas", which is from the package bos.perf.tools.

Issue "lslpp -l bos.perf.tools" to see which version of this fileset is installed if any and invoke it with "topas".

Alternatively you could use "nmon", which is similar to topas, but not officially supported by IBM. Another alternative would be "monitor", which is also not officially supported, but can be downloaded from the Bull-site. Invoke it with "monitor -top". At least one of these tools should be installed on every system. If it isn't - hang your SysAdmin to the next lantern pole for proven indolence.

I can assure you the machine *will* survive a reboot even with a full /tmp filesystem, there will be no problems because of this.

The .map-files are presumably ASCII files containing the layout of the logical volumes (more or less: filesystems) on the disks. Delete them, you can easily create them anew with "lslv -Lm <LVname> [> <file>]".

If you have to increase the size of the /tmp-filesystem:

Issue "lsvg rootvg" to get the PP size (in the right column near the top). You can increase the size in multitudes of this unit. Lets say it is 32MB. If you want to add 128 MB to the filesystem you will have to add 4 such partitions (4x32=128). Don't care about mirroring or so, this is handled by the logical volume manager automatically. Per default /tmp is on the LV hd3. Issue

# extendlv hd3 4

This will increase LV hd3 by adding 4 logical partitions (if the LV is mirrored this would add 8 partitions physically on different disks, yielding 4 partitions logically, or even 12 PPs on 3 disks if there are two copies).

Now that you have increased the LV you need to increase the FS to make use of the additional room. This time you have to specify the space in blocks, which are 512 Bytes big. First we calculate how many blocks the 128 MB are:

# print - "4*32*1024*2" | bc
262144

The result is used to increase the filesystem:

# chfs -a size=+262144 /tmp

The filesystem is now increased by 128 MB. Don't bother to umount the FS, this is not necessary. You can do all that while the applications are running without any problem.

Hope this helps.

bakunin