Hi
My cluster.log file is eating up most of the space in /usr filesystem. I have an active passive HACMP cluster (version 5.4).
cnpsit02 [/usr/es/adm] #du -sg * | sort -nr
1.72 cluster.log
cnpsit01 [/usr/es/adm] #du -sg * | sort -nr
2.32 cluster.log
Please let me know how to solve this. Thanks.
if your cluster.log is file and not a directory, maybe something like
# make a copy of your log
cp /usr/es/adm/cluster.log /tmp/cluster.log
# zip the copy to avoid wasting too much space
gzip /tmp/cluster.log
# keep temporary the 300 last line of your logfile
# put it back to your logfile in a replace (>) manner (not a append >> manner) AND using a cat
# so you won't loose the inode of the logfile so the process that is currently dealing with your logfile won't get fooled.
tail -300 /usr/es/adm/cluster.log >/tmp/cluster.log.tmp && cat /tmp/cluster.log.tmp >/usr/es/adm/cluster.log
# cleanup the temporary file you've used
rm /tmp/cluster.log.tmp
Using /var for logs is one of the most common mistakes when configuring an HACMP cluster. This is a very long story that dates back to at least HACMP V4 and it is very sad that IBM did not manage to get around that problem for years now. So it is the cluster admin's responsibility to not do that. Search this group for questions related to /var getting full and how to avoid that.
You move the cluster logs positions with cluster commands only. Read the cllog manpage. There is also a CSPOC panel if you prefer SMIT.
To take the pressure from the /var filesystem you could remove (i.e. mv and/or rm) unused logs (the rolling logs have a trailing number with 1 being the youngest and 7 the oldest log). Some logs are being held open by cluster processes. You must not move them but you could blank them with
# > big.log
or if you prefer the non-AIX long version
# cat /dev/null > big.log
This way you can prevent your server from dying a painfull and slow death. If you think you still need the log's content just copy it to any place you like before blanking it.
I already answered this question in your previous post http://www.unix.com/aix/148629-files-can-deleted-usr-aix-hacmp-node.html
on both servers create a file system for the HACMP logs and configure HACMP to log there. and then setup a rotate job.