Hi All!
I need to have a script that counts the number of files arriving in a landing directory, them some app pick these files to be processed and load to a DB. But this process is so fast that I am not able to count all the files arriving on a landing directory.
Please can you help?
My initial idea was to insert in crontab a script that every minute does a
ls | wc -l
and outputs this number to a file, but this is not catching all the files arriving
A count is something that makes sense because you have put down rules that make it possible (it becomes static...) e.g. "at an instant T" or " during an elapsed time of T1".
Here you mention nothing of the sort...
Did you analyse what ? Is it the count not following - or - the next process taking so much time when it finishes new files have already arrived?
the files are being sent to a landing directory, and from there a process picks them up, and it moves to somewhere else.
what I want to count is the number of files arriving in this landing directory, before they are moved away
what process picks up these files and how is this triggered? You might want to look in that direction to be able to count the files on time
One possible alternative is to use another intermediate "landing" directory for the counting task.
Another alternative leverages a kernel event monitoring interface. For example, on Linux, there's inotify, which a shell script can use to monitor file creation via inotifywait.
Regards,
Alister
Unfortunately, I dont know what processes picks up the files, this is a comercial application application that does that
you should use the below command to count the files.
ls -l | grep "^-" | wc -l
ls | wc -l
the above command will not count the proper number of files since ls command displays the output in a single line one after the other.