(w)get web server's directories + bash script

Hi everyone.

I need to write a script which will download files/folders (a huge collection) to the local file server (centOS 4.4 Server),
and check regularly (every 6 hours or so if any new files are present, or if the old ones were modified to update contents).

Any insights on how to tackle this problem is greatly appreciated.

Thx.

sounds like the perfect job for rsync

rsync - Wikipedia, the free encyclopedia

Thanks for the reply, it certainly gave me an alternative how to handle local synch of files; however, I need to get the files through (authed) http off the internet.

What would be the easiest way to do it?