i have directory for routers backup files from different dates multiple date\ip-address files
how do i keep only last 10 for each router IP address for example only 3 backup for each IP
Different people will have different solutions for housekeeping jobs like this.
If you are generating backup files on a regular basis (like one per week for example) my approach would be to have a script that calculates a timestamp for n days ago and then use 'touch' to amend the timestamp on a timestamp file that you keep in that directory. Then use 'find' with a '!-newer ' switch to find and delete all files older than the calculated timestamp.
This way it doesn't matter how many router backups you are trying to rotate and saves having to manipulate specific filenames/dates.
Other users on this site may well have other ideas.
(Replace the printf command by an rm to really delete them.)
But awk cannot get the file sizes. You can run your find command in addition; this is an OR condition.
Another way, for an AND condition, is ls -s and a filtering while loop:
ls -sr *.txt | awk -F_ '++cnt[$2] > 10' | while read -r blks fn; do [ $blks -lt 3 ] && echo rm "$fn"; done
hello
the backup files are creating on regular basis (like one per week)
i have 5K devises to backup some times i cannot reach them that is the reason that i am deleting files small them 3000b
i need to keep last x files from each IP
thanks
hello
the backup files are creating on regular basis (like one per week)
i have 5K devises to backup some times i cannot reach them that is the reason that i am deleting files small them 3000b
i need to keep last x files from each IP that is the reason
why i cannot del file via -mtime +10
some time i cannot access device for 3 weeks or more andi need to keep the last x files (with more then 3000b size) for this device
thanks
I tested a quick Ruby script for you, but not completely because I do not want to spend time creating files of various sizes, sorry:
Here’s the equivalent script written in Ruby, partially tested only:
Ruby Script (First Draft)
require 'fileutils'
# Directory containing backup files
backup_dir = '.'
# Maximum number of files to keep per IP
max_files = 2
# Step 1: Filter files larger than 3000 bytes and group by IP
files_by_ip = Dir.glob(File.join(backup_dir, '*.txt'))
.select { |file| File.size(file) > 3000 } # Filter by size
.group_by do |file|
file.split('_')[1] # Extract the IP (second part of the filename)
end
# Step 2: Sort files by modification time for each IP and keep the newest `max_files`
files_to_delete = []
files_by_ip.each do |ip, files|
# Sort files by modification time (newest first)
sorted_files = files.sort_by { |file| File.mtime(file) }.reverse
# Collect files beyond the newest `max_files`
files_to_delete.concat(sorted_files[max_files..]) if sorted_files.size > max_files
end
# Step 3: Delete extra files
files_to_delete.compact.each do |file|
puts "Deleting: #{file}"
FileUtils.rm(file)
end
puts 'Cleanup complete!'
How It Works
Filter Files by Size:
File.size(file) > 3000 ensures only files larger than 3000 bytes are considered.
Group Files by IP:
group_by { |file| file.split('_')[1] } groups the files based on the IP address extracted from the filename.
Sort and Retain Newest Files:
Files for each IP are sorted by their modification time (File.mtime) in descending order.
Files beyond the newest max_files are added to the files_to_delete array.
Delete Extra Files:
FileUtils.rm(file) deletes the files that are no longer needed.
Change max_files = 2 to adjust how many files per IP are retained.
Replace backup_dir = '.' with the path to your backup directory.
Running the Script
Save the script as cleanup_backups.rb and run it:
ruby cleanup_backups.rb
Sorry, it's only a draft for you @sharong and not a complete solution, so you need to further test it and then for more enhancements, please go for it.
My 2 cent:
Add a purge section at the end of your backup routine so that, after each router backup, it would list files with the the router name/IP as a prefix, sort by date, select all but last 10, filter out the too small, then remove.
This would have 2 advantages over a general script which would purge files for all devices:
No need to fiddle with router name/IPs and backup dates at the same time to filter files. The backup routine already has name/IP, so the purge routine only has to deal with dates.
Purging happens right after a new file gets created, minimizing the presence of excess files.
My bet would be the following, using bash, as requested by OP:
BKPDIR=/where/my/files/are
cd $BKPDIR
ls | cut -d"_" -f1-2 | sort -u | while read PATTERN
do
FILESTODELETE=$( ls -t ${PATTERN}* | tail +10 )
echo "Deleting $FILESTODELETE"
echo "$FILESTODELETE" | xargs rm
done
As far as I see, the backup file names are formatted like UNIT_IP_DATE.txt so I list them and filter them by the first two underscore separated fields. Then I go through all the UNIT_IP pairs and list the files in timely order to delete all but the first 10.
Although the logrotate or time based solutions might be better.
tail +10 would cause it to delete all but first 9 files - the N-th line is always included in the output of tail +N.
When working with \n delimited inputs such as this, it's better to use xargs -L1,...
which is still not ideal, because if there ever are fewer than N files present in the directory (matching the pattern of first found file), this will cause rm to attempt removing unnamed files / empty lines (basically returning with error rm: missing operand), as the FILESTODELETE strings created by command substitution will be "" in this case. Instead of such command substitution (containing | tail +N), I'd go with an array, and remove only files represented by values from N-th index upwards (and only in case an array for specific pattern contains more than N elements).