Folder level size monitoring

Hi All,

I have a requirement to monitor the sub-directories under /home in a way that if the the folder size increases by 30 GB in a span of like an hour then it needs to send email alerts listing what as the actual size was and what's the current size which the subject listing the sub-directory name. Can anyone help me with this script. It's a redhat server. Have tried few scripts from this and all give some or the other error which I cannot comprehend since I am totally totally new to scripting.

Thanks,

Not tested .. this code is just for this scenario ..

while [ 1 ]
do
        find /home -type d -size +30000000 -print > find_contents
        [ -s "find_contents" ] && mailx -s "Directories having 30GB more size" abc@xyz.com < find_contents
        sleep 3600 ## sleep for 1 hr
done

Hi Jayan,

Thanks for the script. I need to convert this into a bash script...how do I do that?

Regards

Put the above code in a file and run it ...

If you are looking for differences since last run of over 30Gb rather than just current size, you could probably write a process using the output from the du command, assuming that it is available in your OS.

Depending how you want to monitor, you could try something like (in ksh):-

#!/bin/ksh

current_file=/tmp/current    # Pick what you like here
previous_file=/tmp/previous

mv $current_file $previous_file
du -ks /home/* > $current_file
paste $previous_file $current_file | while read prev_size dir1 curr_size dir2
do
   ((delta_size=$curr_size-$prev_size))
   if [ $delta_size -gt 3072000 ]    # I think that this is 30Gb~ish
   then
      # Take some action
   fi
done

Does that help?

Robin
Liverpool/Blackburn
UK

Hi Jayan,

Have tried you code....but when I run it....it seems that that code does not stop running. After running...it does not give me back the shell prompt. Not sure whether I am doing it wrong here.

Hi Robin,

I will test you code and let you know.

Thanks

Shailesh6,

Jayen's code is in an infinite loop.

Mine does require that you are able to read all the subdirectories of /home, and their sub-directories, and their's, and theirs ..........

Perhaps a tool like sudo for the du command would be good to elevate your privileges for the execution of that alone.

Robin
Liverpool/Blackburn
UK

Robin,

If I want to mail the alert condition to a email address what would my code look like....I mean can you replace the Take Some action with some code that would send an alert to an email address with a particular subject. And with current_file=/tmp/current and previous_file=/tmp/previous am I supposed to given a directory path that I want to monitor? I want to do it for all the directory under /home and I am performing this testing with root user.

Thanks

For the e-mail, it depends what you want to send and who to.

The files mentioned are just files for this script to run and do it's work. The key bit is being in the directory/filesystem you want to be monitoring, in this case it's /home as in listed in the du command.

If the email is to the directory owner, then it depends if you have an e-mail address (or could build one) for each user. Perhaps you have it as the user comment so you could extract it from /etc/passwd (usually field 5 I think) Once you have determined the e-mail address, your command could be something like:-

......
if [ $delta_size -gt 3072000 ]    # I think that this is 30Gb~ish
   then
      echo "You have a quickly growing directory called $dir1\n\nThe administrator." | mail -s "Subject line in here" $email_address
fi

If your email is not based on the same server, then you will need to configure the /etc/sendmail.cf file, however this probably just means setting a DS record (which is commented out by default) to name your e-mail server or some relay that will accept smtp and route it on for you. Find the line that is for the DS statement and uncomment it, adding in your mail server, e.g.:-

DSyour.mail.server

Make sure that the address can be resolved (DNS prefereably) and that you can open port 25. If you try:-

telnet your.mail.server 25

... then you should get a HELO reply rather than a timeout.

Does this help a bit more?

Robin
Liverpool/Blackburn

Hi Robin,

This is help a lot lot more...my email is not based on the same server so I will need to make change in the sendmail.cf. I will check this an let you know.

Thanks.

Caveat: That approach will silently implode if files or directories are added to or removed from /home.

Regards,
Alister

Yes, agreed & point taken. I suppose we could cover it off by checking that "$dir1" = "$dir2", but then how you carry on could be a nightmare. :eek:

The alternate would be to read the first file and then get the approptaite record from the second, but that's a much bigger thing to think about.

Anyone facy taking it on? :wink:

Robin

Hi Guys,

Please give me brief how the original script given by Robin would cause any implode?

Thanks

You would probably learn more if you tried it for yourself. Run the script. Then delete and/or add a directory. Run the script again. Perform the test with pathnames that occur in early in the sort order for maximum effect.

Since those are kilobytes, that's actually closer to 3 gigabytes.

Regards,
Alister

1 Like

Ah yes, never good at counting zeros. I agree, that's about 3Gb.

:o Sorry about that. :o

A change in content of /home could cause a spectacular failure, but as I'm not a software house, this is just some help to get started. Best to consider the 'interesting opportunity' for yourself to work out how you want to detect / handle this exception.

I'm not sure what happens at the end when the two data files that are pasted are different length. You would have to try and adjust the scripting suggestions. It might go a bit wild.

Take care,
Robin
Liverpool/Blackburn

Hi Robin,

I am still not convinced how this script to cause any failure or trouble. From what I understand the script is going to check the du for home...report the size of each sub-folders...dump to current file and previous file...make a comparision between these two files...find which folder has 30 GB more data from the time the script was last run...and report any alert for such folders. Not sure am I missing out here? Sorry

Thanks

Well, in /home on a test server I have, the du reports only a few lines:-

8     caroot
0     guest
0     lost+found

If I add an empty directory aaaaaaaa then capture the output from du -ks * to a file to use as my initial 'previous' file, then remove the directory aaaaaaaa again, running the script tell me that directory aaaaaaaa (which no longer exists, of course) has grown by 8, and caroot (which has not changed) has shrunk by 8. The others are unchanged, but it could get complicated if they had any contents to start with.

Try it out as a test (i.e. don't send any e-mail) It is just a report, after all.

Does that help to illustrate the risks? Of course, if you are sure that there will be no changes at the top level (e.g. new users) then go ahead.

Robin
Liverpool/Blackburn

1 Like

Hi Robin,

The script is working the way I desire. However, the email alert that is being generated looks like below:

Total amount of data in is KB and exceeds the limit of kB!

Our original script had "Total amount of data in $dir is ${size}KB and exceeds the limit of $limit kB!" .....how can I get the name of the actual directory name (that exceeds the limit) and the size in the alert email?

Thanks

---------- Post updated 10-06-11 at 02:19 PM ---------- Previous update was 10-05-11 at 09:56 PM ----------

Hi Robin,

Can you help me with my last post for the alert email. Thanks.

Shailesh

Ah! Sorry.

The variable in the message should be $dir1 because that is what is being read back in from the work file. Of course, you could user $dir2 too, because they should always match, unless we have that problem where files/directories change in /home

Oops! :o

Apologies again,
Robin
Liverpool/Blackburn

Hi Robin,

Thanks for the reply. Can you let me know whether we can also report the size of the folder along with this script. I need to have the alert saying that folder ABC current size is xx KB/MB and earlier it was xx KB/MB. Also, I need to get down deeper like it can check for more sub-folders within the /home/ABC folder...can that be done?

Regards.