What to do when mtime doesn't work?

I have a folder that I need to search for new files and copy on the latest. I've been using "-mtime -1" in my command line but it doesn't seem to work.

I've been meaning to fine a different script because files are dropped into the folder all day long and because of the -mtime, I've only be able to look for files once a day.

Does someone have a better way for me to search this folder throughout the day to find and copy only the latest files?

can you post the full command line?

I'm guessing you're using 'find'.

Have a cron job that does a listing of the directory and compares the results with the previous listing?

You can also use "ls" to sort files by timestamp.

Or touch a file at the end of each collection and simply gather the files newer than the file you touched the next time, then touch the file again.

Here's the command:

find . -type f -name "raw_file_*" -mtime -1 | cpio -iv <new directory>

The problem is, when I run this it copies ALL the files in the directory instead of those that are created in 1 day or less. It's crude, cause what I really need is something to look for only NEW files and copy those.

Many versions of the find command have an option called -newer which can be used to find files which are newer than the date of a reference file. See the find(1) man page for further information.

So at the end of the script, have the script touch the same file again?

How do files appear in the directory, are they copied or moved/renamed.

If copied, then how are you going to avoid copying a half copied file?

Somethine like this?

find . -type f -newer afterme.txt | cpio -dumv <new directory>
touch -t afterme.txt

They're moved from a server to another server. It's the second server that I need to copy the files from.

Touch the file once manually. Then after you have found the list of files touch it again.

manually:

touch my_touch_file

Then in the script:

find <dir> -newer my_touch_file > listoffiles.txt
touch my_touch_file
cpio -pdumv /somewhere/else < listoffiles.txt

or do it in the background:

find <dir> -newer my_touch_file | cpio -pdumv /somewhere/else &
touch my_touch_file
wait

Save the list fist, and then feed that into cpio, otherwise you could get a file that will arrive while you are doing the cpio and you'll never retrieve it.

Porter has a point which I was also wondering about, how can you be sure you won't get a partial? If it is longer term storage it doesn't matter since you will get them next time around, but if the files are being processed then you could end up processing an incomplete file.

Not if the following happens

1. start copying the file into the directory locally

2. find picks up the file and starts copying to remote server

3. copy to remote server finishes (that was started in 2)

4. copying of the file locally finishes (that was started in 1)

5. touch occurs

so the file would be older the than the timestamp file and hence not get picked up next time.

Possible but very highly unlikely if the second option above is followed, or an extension of using two touch files on a round robin and touching before find could be used that would definitely not fail.

Either way it is simpler and more efficient than comparing directory listing.

Thanks for the help. I'll give the suggestions a try.

The pre-touch file method.

1. Manually create a file older than the last unretrieved file using "touch -t"

touch -t <MMDDhhmm> my_old_touch_file

touch my_new_touch_file
find . -type f -newer my_old_touch_file \! -newer my_new_touch_file  | cpio ....
mv my_new_touch_file my_old_touch_file

If I understand the problem correctly, you only want to copy new files.

1). Manualy do a list of your directory and direct it to a file.
ex:
ls > filelist.txt
2). In your script, replace the find with this.
ls > filelist_new.txt
**** The below should appear in a loop
diff filelist.txt filelist_new.txt | grep -v 'file2' | grep '>' | awk ' { print $2} '
*****
example:
for FileToCopy in `diff filelist.txt filelist_new.txt | grep -v 'filelist_new.txt' | grep '>' | awk ' { print $2} '`
do
cp $FileToCopy <where ever it goes.
done
cp filelist_new.txt filelist.txt

Every time you run that script, you will only pick up new filers.

;o), I only tested the diff part of this script. But, it should work. This is sh shell scripting and not bash.

If your moving files, after I think about all the posts... This wont work, and I appoligize for the dumb post.

droolin

Not moving files, copying them to another location for pickup by another script.

These are good guys! I'm starting to love this forum

your other option is rsync. that's an excellent tool that allows you an unbelievable amount of control.

Hi can some one tell me the alternative for
-mmin command?
Its not working in SunOS

"find . -mmin -30". This works in other machines but not in SunOS.

Please help

Robin Paulose

-mmin -30 means 30 or less minutes.
Use touch to set an mtime on a dummy file - the mtime being 30 minutes ago.

touch -t 200903170800 dummy
find . -type f -newer dummy