/var size is increasing day by day

Hi experts,

I am facing a big problem. i use solaris 9. i found size of /var is increasing day by.
snapshot of a yester moring-
/dev/vx/dsk/var 15G 14G 1.1G 94% /var
snapshot of a yesterday everning-
/dev/vx/dsk/var 15G 14G 824M 95% /var

I am expecting /var/adm/messages size is increasing and hence /var size is increasing. Its my assume. But I don't know how to delete lot of records from the file /var/adm/messages

Please guys advise me asap. I am danger situation and i am afraid.

Morning Snapshot- /var/adm

bash-2.05$ ls -lht
total 1250600
-r--r--r-- 1 root root 27K Nov 2 09:55 lastlog
-rw-r--r-- 1 root root 109M Nov 2 09:55 messages-rw-r--r-- 1 root bin 15K Nov 2 09:31 utmpx
-rw------- 1 root root 42K Nov 1 13:05 sulog
-rw-r--r-- 1 root root 179M Oct 29 03:10 messages.0
-rw-r--r-- 1 root root 123M Oct 22 03:10 messages.1
-rw-r--r-- 1 root root 87M Oct 15 03:10 messages.2
-rw-r--r-- 1 root root 112M Oct 8 03:10 messages.3
-rw-r--r-- 1 root root 0 May 10 2005 vold.log
drwxrwxr-x 5 adm adm 512 May 10 2005 acct
drwxrwxr-x 2 adm sys 512 May 10 2005 sa
drwxr-xr-x 2 root sys 512 May 10 2005 sm.bin
-rw------- 1 uucp bin 0 May 10 2005 aculog
-rw-rw-rw- 1 root bin 0 May 10 2005 spellhist
drwxr-xr-x 2 adm adm 512 May 10 2005 exacct
drwxr-xr-x 2 adm adm 512 May 10 2005 log
drwxr-xr-x 2 adm adm 512 May 10 2005 passwd
drwxr-xr-x 2 root sys 512 May 10 2005 streams
bash-2.05$

Evening snapshot- /var/adm

bash-2.05$ ls -ltrh
total 1269464
drwxr-xr-x 2 root sys 512 May 10 2005 streams
drwxr-xr-x 2 adm adm 512 May 10 2005 passwd
drwxr-xr-x 2 adm adm 512 May 10 2005 log
drwxr-xr-x 2 adm adm 512 May 10 2005 exacct
-rw-rw-rw- 1 root bin 0 May 10 2005 spellhist
-rw------- 1 uucp bin 0 May 10 2005 aculog
drwxr-xr-x 2 root sys 512 May 10 2005 sm.bin
drwxrwxr-x 2 adm sys 512 May 10 2005 sa
drwxrwxr-x 5 adm adm 512 May 10 2005 acct
-rw-r--r-- 1 root root 0 May 10 2005 vold.log
-rw-r--r-- 1 root root 112M Oct 8 03:10 messages.3
-rw-r--r-- 1 root root 87M Oct 15 03:10 messages.2
-rw-r--r-- 1 root root 123M Oct 22 03:10 messages.1
-rw-r--r-- 1 root root 179M Oct 29 03:10 messages.0
-rw------- 1 root root 42K Nov 2 09:59 sulog
-rw-r--r-- 1 root bin 15K Nov 2 18:24 utmpx
-rw-r--r-- 1 root root 118M Nov 2 18:29 messages-r--r--r-- 1 root root 27K Nov 2 18:29 lastlog
bash-2.05$

Best Regards,
Purple

How are the messages.0, messages.1 etc files created, are they being created automatically? Or do those dates match reboots. You can archive and delete the old message.* files.
There are various "log rolling" solutions available.

i don't know how messages.0, messages.1 , .2, .3 has created. System is up at least for last one and half months. So, those dates should not be match with reboots. If you see messages.0,1,2,3 size has not changed. Only "messages" size has changed. I m not sure Is only for file 'messages'- the size of /var is increasing?? Or else something hidden.
Could u pls tell me about something 'log rolling'??

Probably this may help:

HOWTO Rotate logs on Sun Solaris - BeezNest

Hi
The first thread in this Forum is named as following:

Filesystem full - what to look for << Very useful!!!!

Please first read this thread and then you can say what is the reason that the /var file system is so full.

In my opinion the messages files are not the reason!!
(But to use the "log rolling" is very comfortable.)
The article shows you how to handle this situation.

This is my favorite command to find problematic files and directory's
du -akd /var | sort -nr | more

Best regards joerg

H Joerg,
Thanks for the suggestion. I tried with du -akd /var | sort -nr > /tmp/duvar.out. But i don't know how to find the porblemetic files and directorys from there. In the output file
is see somthing like below. left value i thing telling the size. But what do i think to c these???
...
21104 /var/sadm/pkg/VRTSvxvm/save/116697-03/undo.Z
16042 /var/mail
16000 /var/mail/bgw

Hi,
for more human readable:

But you must use a ksh!
and
"M " is not a space after the "M" it is a control-I (It is a big I like Imagine (Tab is the same))

du -akdh /var | sort -nr | grep "M "

With this you can find the biggest files and the dir with the most among of filesystem place.
This search only Megabyte big files if you looking for a Gigabyte file you have to change M to G.

Best regards
joerg

could you please open the message file and see what message they are? I hope it will help you to do RCA.

Hi joerge,

i tried ur suggestion in little different way- "du -akdh /var | sort -nr > /tmp/var.out". It works well. Can you tell me also can I see when file has last modified by "du -akdh /var | sort -nr" Is it possible somehow to accomodate 'ls -lhtr' with command du -akdh /var | sort -nr | grep "M ". ??

Hi ThePurple

The fact that your messages.x files are so massive tells me you are logging shedloads of stuff in there.

I suggest you make copies of the messages.0 - messages.4 files to an alternative location and then do a # cat /dev/null > messages.0 etc to empty them. This will save you a few hundred MB and give you some breathing space. The next step would be to change your log rotation to rotate your messages files when they get to 10MB or something. Afterwards you should have a look at your /etc/syslog.conf file and amend it to log more efficiently.

You can use this command :

#for file in /tmp/var.out; do ls -lrt $file; done --> it will show you the timestamp.

If the file /tmp/var.out contain only file name, then use

#for file in /tmp/var.out; do find / -t f -name $file -print; done

Hi, you might be can check the /var/adm/sa. did you turn on the sa monitoring.. Just a piece of info.

Regards
Dunstan:rolleyes: