Removing folders - hierarchical stored

Shell Programming and Scripting
1

my data files are stored in such way like year[parent folder]->month->day-> data files, my requirement is to find out number of days of files from today to the starting day of storig files.
root directory = work/user/year[parent folder]->month[Child folder]->day[grand child]-> data files
structure is as follows

 
2013->05->as of today there are 01 ,02 ,03 etc todays date that is 15  [tomorrow 16 will be added and goes on]
2013->04->01,02,03......30
2013->03->01,02,03......30,31
2013->02->01,02,03......28 
2013->01->01,02,03
2012->12->01,02,03......30,31

.
.
2012->01->01,02,03......30,31
2011->12->01,02,03......30,31
.
.
.
2011->01->01,02,03......30,31
2010->12->01,02,03......30,31
.
.
.
2010[year]->01[january]->01[day],02,03......30,31
..
..

.

I want to find out how many days of data files are stored, if it exceeds 541 days , remove oldest folder ones to make total number of days starting from today is 541.

Can you please give me most efficient code to execute it ?

Thanks

2

Assuming the files in a certain day folder are all created the same day,
a simple find on the file dates should work:

find work/user/[1-9][0-9][0-9][0-9] -type f -mtime +541 -exec rm {} \;
3

but my folder structure is

 
 
2001-2100/01-12/01-30/files

If we remove dates in removal process based on 541 criteria , 1-30 , then automatcally month folder should be removed,

in removal process, if there is no month folder, then year folder must be removed automatically

Thanks,

---------- Post updated at 06:47 PM ---------- Previous update was at 06:02 PM ----------

I changed retention criteria from 541 days to 455 days , we need to remove all parent and children folders and files inside date folders from from feb 15, 2012 to older dates . but 2012 and feb month folder will be there untill we hit 455 lmit on feb 01 ,2012.

so if we calculate , we need to remove all files in 2010/01-12/01-31 folders and 2011/01-12/01-31 foders and in the year 2012, we will keep till feb 03 and 15 days of files in feb to make it 455 days

[RIGHT][/RIGHT]

---------- Post updated at 06:57 PM ---------- Previous update was at 06:47 PM ----------

it should consider backward when counting number of dates, 2013/may, 2013/apr,2013/mar,2013/feb,2013/jan , 2012/dec, 2012/nov, 2012/oct, 2012/sept,2012/aug,2012/jul, 2012/jun, 2012/may,2012/apr,2012/mar,2012/feb , then keep files inside of folder date from 28 to 13 to retain 455 days of files. file exist only in date folders.

---------- Post updated 05-16-13 at 02:04 AM ---------- Previous update was 05-15-13 at 06:57 PM ----------

I want to add a few things to addition to above mentioned points. This might help you understand my requirement more clearly

Designing criteria:

  1. its desined to hold files in hierarchical folder structure of year/months/dates/files
  2. root folder contain year folders
  3. Year folder contains month folders
  4. month folder contains date folders
  5. date folders contain data files
  6. date folder contains only the data files that particular date produced.
  7. remove all data files which is older than 455 days
  8. remove all date folders which is older than 455 days
  9. remove all months folder if they do not contain any date folders due to 455 days removal process
    10.remove all year folders if they do not contain any months folders due to above process
  10. script should consider from 2000 to 2100 years or dynamic
    it should be removed as the following direction, for example considering today is may 15 2013,
    then folder 2013/05/15 to 2012/02/29-14 should be retained and rest of the olders date, months and years folders
    should be removed
 
 
YEAR MONTH DAY
2013 5 15
2013 4 30
2013 3 31
2013 2 28
2013 1 31
2012 12 31
2012 11 30
2012 10 31
2012 9 30
2012 8 31
2012 7 31
2012 6 30
2012 5 31
2012 4 30
2012 3 31
2012 2 29-14
=======================================================
2012 2 1-14
2012 1 31
2011 12 31
2011 11 30
2011 10 31
2011 9 30
2011 8 31
2011 7 31
2011 6 30
2011 5 31
2011 4 30
2011 3 31
2011 2 28
2011 1 31
2010 12 31
2010 11 30
2010 10 31
2010 9 30
2010 8 31
2010 7 31
4

No need to post new post every time to add info, use edit button.
What have you tried so far?

5

If I use this command, does this remove the files in a given directory or it recursively go in each parent-child-grandchild diretory and delete the files and remove the directories those are empty?

As -type f - stands for finding only files , will it not consider directories as well?

-exec rm - stands for removing only files , but not empty folders.

how do we do it dynamically go in each folders recursively and remove files that are older than 455 days and remove the empty folders?

6

find will recurse thru all subfolders (of the given start folder) - not very efficient.
And seems risky; do a dryrun first: -exec echo rm -f {} \; i.e. put an echo before the rm

7

do you recommend any other solution other than find? Why did you say "find" is risky and not efficient?

If I put echo before rm -f what will happen?

8

The echo displays the generated rm commands - instead of running them.