First time poster. I am looking for a way to script or program the process of moving files from one folder to another, automatically, based on the count of files in the destination folder.
I was thinking a shell script would work, but am open to the suggestions of the experts here.
Files I am trying to move:
UNIQUEID.rmdcall
Files start at:
/tmp
Files need to go to:
/tmp/test
Details:
If the count of .rmdcall files in /tmp is greater than 0, then count the .rmdcall files in /tmp/test, otherwise do nothing.
If the count of .rmdcalls in /tmp/test is less than 30, then move up to 100 of the oldest .rmdcall files from /tmp to /tmp/test, otherwise do nothing
Repeat the process every 5 seconds
Thanks for any help you can provide, it is greatly appreciated!
Please review the rules, which you agreed to when you registered, if you have not already done so.
If you did not post homework, please explain what the problem you have described is intended to do. Needing to move 1200 files per minute from one directory to another is not a common requirement. Why don't you just have whatever is creating these files in /tmp create them in /tmp/test instead?
Furthermore, always tell us what operating system and shell you're using when posting requests in the Shell Programming and Scripting forum and show us what you have tried to solve this problem on your own.
This is not homework, it is an idea for a more automated workflow. We have a folder (/tmp/test) that an application listens to. When a file is moved into that folder, it processes and deletes the file. Basically I have work to do in the /tmp files and I need to throttle how much I send to the application /tmp/test so I do not crash the application. This is done manually today (someone moves the files to /tmp/test periodically). I am hoping we can automate this to save time.
It is not that 1,200 files per minute will move but up to 1,200 files may move (if they exist).
There is another process in /tmp/test that processes any file dropped in that folder. The problem is that I cannot move all the files at once or I risk overloading the workflow. The goal is to systematically throttle the files coming into /tmp/test to avoid this.
Linux Version: Sangoma Linux release 7.4.1710 (ID_LIKE="centos rhel fedora")
Shell Version: GNU bash, version 4.2.46(2)-release (x86_64-redhat-linux-gnu)
I don't have a lot of experience in scripting, but I know I need the mv command to move the files. I think I need the touch command to dictate when to move the files.
I also know I can get file counts by using the ls /tmp/*.rmdcall |wc -l command.
I am just not sure how to string these all together, and time is currently not on my side. Any insights? Thanks in advance for any help you can provide.
#!/bin/bash
src="/tmp"
dst="/tmp/test"
threshold=30
file_count=$(ls ${dst}/*.rmdcall 2>/dev/null | wc -l)
if [ $file_count -lt $threshold ]; then
transfer=100
for f in ${src}/*.rmdcall; do
if [ $transfer -eq 0 ]; then
break
fi
[ -e "$f" ] && mv -v "$f" "${dst}/" || break
transfer=$((transfer -1))
done
fi
How to identify the oldest files - by modification time stamp? By UniqueID (if in increasing order)?
EDIT: OK, let's postpone the file identification question for now. I'd prefer a different approach. Your file names seem not to contain any white spaces, so deploying xargs seems safe.
After having determined the file count in /tmp/test having dropped below 30 just like Aia proposed, how about
ls /tmp/*.rmdcall | head -100 | xargs echo mv -t /tmp/test
? Should your system's config parameter LINE_MAX be exceeded with the resulting command line, add the -n option to xargs like e.g. -n50 . Remove echo when happy with the proposed result.