Move folder once it is "ready"

I want to move the folder once it stops receiving data. The sign to tell when the folder is ready is the number of files in it. Here, once the folder has 27 files, it tells me it is "ready". An example, S46 has 62 subfolders with similar pattern S46_005, S46_007 ---> S46_127. Some may not have 27 files, so leave those. Eventually all the 62 subfolders will be grouped into a master folder S46. My BASH code:

for j in S46_*
do
a=$(ls ${j} | wc -l) 
if ( $a == 27) 
then 
mv ${j} S46
fi
done

I always got error:

27: command not found!
27: command not found!
...

It seems related to the variable $a. What did I miss with my code? Thanks!

Try:

if  [ $a -eq 27 ]

However, something to think about:
Once it stops recieving files, and having 27 files is quite diffrent.

hth

1 Like

Ahh, should have spotted that by myself. Mixed up with my C practice these days.
I am aware Once it stops recieving files, and having 27 files is quite diffrent. From my example, the file is small and writing is fast so that I assumed (!) once with 27 files, the folder is ready.
Do you have some scripts for that?
Thanks!

Nope, havent come by such a situation/requirement yet.

27 files is not 27 complete files.

Then, let me ask a related question: How to check if a folder is ready without any further operations on it, like writing files into it for this circumstance? Googled for a while it seems no direct discussion on this.
Thanks!

I'd take an appraoch of repeatingly compare the output of:

du -s /path/to/S46_127

With some delay of course, once the values are identical, the files should be 'ready'.

Maybe this gives another option lsof /path/to/S46_127/*

hth

Polling the folder and hoping that everything is done is asking for trouble. If you actually care about not copying over a half-written file, you need an atomic step.

First of all, the directory that you monitor and the directory which is the landing place for the transfers should not be the same. Secondly, they should lie on the same filesystem. Thridly, when a file arrives on the monitored directory (given the conditions above, it would be a POSIX atomic rename(2) operation), that file is always complete and any counts can be considred completed.

Regards,
Alister

1 Like

It's the application's job to tell you when it's done.

1 Like

Thanks alister and corona688!
It does not make much sense to move the files from the folder before it is ready. I came across this situation when there are many samples (~100 x 60) to process and each case is not processed at the same pace, which makes the folder messy. So, I thought to move those processed samples to get rough idea how many of the samples have been handled and how many threads are freed from the cluster.
However, it was great for my learning purpose on shell scripting. Thank you again!