I have run into a problem where about a dozen machines, all the same x86_64 2.6.12 GNU/Linux. For some reason these machines will fill up their /var partition (10G), because their logs never get rotated... Unfortunately, there is no error message from logrotate (would be in /var/log/messages) and the last time logrotate ran (according to /var/log/logrotate.log) was August 11, 2008.
This unfortunately is somewhat of a puzzling problem, making it feel (however unlikely) that this is actually a problem with cron, and not log-rotate. IE. cron failed first, which made logrotate never happen. I had originally thought that log-rotate just couldn't do its job because there was no space available for the rotation to occur. But even when I gave it enough space, cron just never ran the logrotate script. On top of that, I even added my own "append timestamp to file" script to the cron.hourly, and it never got run either. As a test, on one of the servers, I restarted cron... BINGO, it fixed the whole thing.
I could care less about the fact that I can fix it. I want to prevent this from ever happening in the field. In my lab is one thing, but in the field, there goes all my logs for debugging.
Here is a sanity check I ran (just proving that cron was functioning and stopped functioning - all within the same up-time):
[root@babylon5 ~]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda2 9.5G 3.7G 5.3G 42% /
/dev/sda1 190M 18M 163M 10% /boot
/dev/shm 2.0G 4.0K 2.0G 1% /dev/shm
/dev/sda6 44G 8.4G 33G 21% /stuff
/dev/sda5 9.5G 9.5G 0 100% /var
[root@babylon5 ~]# ls -lut /etc/init.d/crond
-rwxr-xr-x 1 root root 1904 Aug 11 15:14 /etc/init.d/crond
[root@babylon5 ~]# ls -lut /etc/crontab
-rw-r--r-- 1 root root 255 Jul 15 10:19 /etc/crontab
[root@babylon5 ~]# grep "Aug 11 15" /var/log/cron
Aug 11 15:01:02 babylon5 crond[9429]: (root) CMD (run-parts /etc/cron.hourly)
[root@babylon5 ~]# ps auxwww|grep cr[o]n
root 3014 0.0 0.0 60404 1296 ? Ss Jul15 0:00 crond
[root@babylon5 ~]# uptime
13:33:34 up 57 days, 3:15, 1 user, load average: 15.63, 15.37, 15.29
[root@babylon5 ~]# grep "STARTUP" /var/log/cron*
/var/log/cron.4:Jul 14 14:16:44 babylon5 crond[3156]: (CRON) STARTUP (V5.0)
/var/log/cron.4:Jul 15 08:53:45 babylon5 crond[3008]: (CRON) STARTUP (V5.0)
/var/log/cron.4:Jul 15 10:19:35 babylon5 crond[3014]: (CRON) STARTUP (V5.0)
[root@babylon5 ~]# tail -n 1 /var/log/cron.4
Jul 20 00:01:01 babylon5 crond[15555]: (root) CMD (run-parts /etc/cron.hourly)
[root@babylon5 ~]# tail -n 1 /var/log/cron.3
Jul 27 01:01:02 babylon5 crond[27941]: (root) CMD (run-parts /etc/cron.hourly)
[root@babylon5 ~]# tail -n 1 /var/log/cron.2
Aug 3 01:01:01 babylon5 crond[17755]: (root) CMD (run-parts /etc/cron.hourly)
[root@babylon5 ~]# tail -n 1 /var/log/cron.1
Aug 10 01:01:02 babylon5 crond[29697]: (root) CMD (run-parts /etc/cron.hourly)
[root@babylon5 ~]# tail -n 1 /var/log/cron
Sep 10 13:01:01 babylon5 crond[22186]: (root) MAIL (mailed 146 bytes of output but got status 0x007f )
[root@babylon5 ~]# ll /var/log/cron*
-rw-r--r-- 1 root root 138502 Sep 10 13:01 /var/log/cron
-rw------- 1 root root 14407 Aug 10 01:01 /var/log/cron.1
-rw------- 1 root root 14581 Aug 3 01:01 /var/log/cron.2
-rw------- 1 root root 14483 Jul 27 01:01 /var/log/cron.3
-rw------- 1 root root 14513 Jul 20 00:01 /var/log/cron.4
Any help?