I am trying to delete all log files in given directories on the NAS filesystem which are older than 4 years including current year. What is basically needed is to pick up current year and then calculate years from where it should be deleted
@patil , Welcome, have you made any attempt, if so please share, if not check out the find man page for details
Example ... look for all bash scripts older than 4years (day granularity)
find PATH -name '*.bash'-mtime <number of days> .....
where PATH and 'number of days' you need to supply
I was attempting to start with find command as well . In details Scenario would be , I have a log folder with different environment folders . I need to check in each folder if there is any log files which is older than 4 years (including current year) and then delete it. To achieve this , I need to pick up current system year and then calculate year which would be 4 years older and then delete it.
@patil ,
have you read the find man page ? , read the example command given then combine that with the details from the documentation .... its really not that difficult you just need to take 5...10mins to read and understand , then test ...
Calculating the days: 4 * 365 = 1460
And in a find command
-mtime +1460
to filter for an age greater than 1460 days.
find startdir -name "*.log" -type f -mtime +1460 -ls
As startdir you put your top log folder. find will automatically recurse to the sub folders.
The -ls displays what it finds.
If it looks right then you can put an action like
-delete
or
-exec rm -f {} +
to really delete them.
See all find options in the man page:
man find
For Linux there are online man pages also. Google for
man find linux
1 Like
Using below it would delete files from 2021 as well but doesn't want so.
find startdir -name "*.log" -type f -mtime +1460 -ls
I've seen in man page and that's where I'm stuck. Any further help would really be appreciated
I strongly recommend you move the files to a temporary "to_be_deleted" directory before you actually delete them.
That way you can easily examine these to be deleted files and then delete when you are sure.
1 Like
Sure Thank you
not sure what
Using below it would delete files from 2021 as well but doesn't want so.
means.
Show the command you are running and (some) of the output .
(use copy/paste to add to this thread)
tks
No, it would not delete files from 2021, it would only delete files older than 23.04.2020
$ date -d '1460 days ago' +'%F %T %Z'
2020-04-23 19:21:44 UTC
$ touch -d '2019-01-01' oldest
$ touch -d '2020-01-01' older
$ touch -d '2021-01-01' old
$ touch -d '2022-01-01' new
$ touch -d '2023-01-01' newer
$ touch -d '2024-01-01' newest
$ find . -maxdepth 1 -type f -mtime +1460 -exec ls -Gg --full-time \{\} \;
-rw-rw-r-- 1 0 2019-01-01 00:00:00.000000000 +0000 ./oldest
-rw-rw-r-- 1 0 2020-01-01 00:00:00.000000000 +0000 ./older
$
If you see any files from 2021 in your list, then likely one (or more) possible reasons may be:
- You used
-NNNNinstead of+NNNNwhen specyfying themtimeparameter - The date/time setting of your system (either the one storing files and/or the one where
findcommand is invoked) is incorrect, i.e. not aligned with the actual, current date/time - Your files are misnamed / their timestamps are mismatched in regards to their names (e.g. names of files from 2020 contain
2021and are misleading you into thinking they're from 2021) - ...
I think you need to clarify "older than 4 years" and "current year". Using mtime +1460 counts from today, so you are deleting files older than 2020-04-23 10:02:57 right now. I find that rather messy and counter-intuitive.
An alternative interpretation is that you want to delete a set of files in the range (e.g.) 2020-01-01 00:00:00 to 2020-12-31 23:59:59 for each year of interest.
The easiest way to do that is to touch a pair of files with those dates, like touch -t 202001010000 /tmp/myFirst, and use find with those as reference files for -newer and ! -newer.
find has a -daystart option that aligns times relative to 00:00:00 (the man page does not specify if that is UTC or Local time). I think you might be seeking to do something similar, but like -yearstart (which does not exist).
The code below is way too full of debug, but I hope it illustrates the ideas. It has to push back to the last second of the previous year, because -newer means exactly that. It should automatically recurse through your sub-folders.
I have GNU bash, version 4.4.20(1) -- hope we are compatible.
The reference files should be in /tmp, because find would probably delete one of them if it found it in your directory. A dry-run without the -delete option is strongly advised.
#! /bin/bash --
#.. Decide the years span.
printf -v Now '%(%Y)T' -1
dtOne=$( printf '%.4d12312359.59' $(( Now - 7 )) )
dtTwo=$( printf '%.4d12312359.59' $(( Now - 4 )) )
#.. Make the reference files.
touch -t "${dtOne}" "/tmp/dtOne${$}"
touch -t "${dtTwo}" "/tmp/dtTwo${$}"
ls -l --time-style='+%Y-%m-%d %H:%M:%S' /tmp/dt*"${$}"
#.. [Debug] Make some test files.
rm ./File.20*.log
for j in {2017..2022}; do
for k in {01010000.00,07231645.25,12312359.59}; do
touch -t "${j}${k}" "File.${j}${k}.log"
done
done
#.. [Debug] Show the test files.
printf '.... Before ....\n'
ls -l --time-style='+%Y-%m-%d %H:%M:%S'
#.. Delete the files in the chosen range of years.
#.. Also print them, sorted (find gets them in arbitrary order).
printf '.... Deletions ...\n'
find . -type f -name '*.log' -newer "/tmp/dtOne${$}" \
! -newer "/tmp/dtTwo${$}" -print -delete | sort
#.. [Debug] Show the ones that are left.
printf '.... After ....\n'
ls -l --time-style='+%Y-%m-%d %H:%M:%S'
#.. Remove the reference files, and prove it.
rm /tmp/dt*"${$}"
ls -l --time-style='+%Y-%m-%d %H:%M:%S' /tmp/dt*"${$}"
4 Likes
find . -mtime +1460 -exec rm -f {} \;
This should work
Then he needs to provide some test input output data, that's what I understood from the original post.
I tried the option with giving mtime parameter but now while deleting the files all of them are not getting deleted from the directory. the command probably exits before deleting all files. Is this due to many large files in the directory? I had to run the script multiple times to clear all required files. Any suggestions?
@patil , as already requested .... statements like 'i tried but it doesn't work', ' ...the files all of them are not getting deleted from the directory. the command probably exits before deleting all file... ' are pretty much useless - why : they're opinion/assumption(s).
That's why we ask for code / example outputs etc - that is typically concise/unambiguous.
I'm using below command in my script
find /data/NAS_Denodo/$value/logs_backup/ -type f -mtime +1209 -delete
tried below also:
find /data/NAS_Denodo/$value/logs_backup/ -type f -mtime +1209 -exec rm {} \;
but the files from 2020 still exists in log directory
hope this helps
There are several suggestions in this topic. Can you show the exact command or script that you used?
This one has several faults:
find . -mtime +1460 -exec rm -f {} \;
(a) It tries to delete all types of "file" -- e.g. directories, links, named pipes, ... . find is a powerful tool -- limiting it to specific types and names is strongly recommended.
(b) It runs a separate rm process for every file -- a serious performance hit. The '+' terminator to the -exec runs batches of files for each rm command.
(c) The -delete option in find is safer and faster than an -exec rm.
(d) It will delete files backdated 4 years from "today", not for an actual calendar year.
As you seem to have now removed all the files as required, repeating this task with a different method would appear to be pointless. That's why we generally test scripts in a "sandbox" before we let them loose on live data.
It looks like the directory gives no rights to group or other, only user.
Deleting a file requires rwx access to the directory containing them (read to find filenames in it, write to amend it by removing the file, execute to search it for lower levels).
So this depends on who ran the script, and who owns the files. A GUI file manager will generally not be as informative as a command line ls -l.