Polling/Interrogate Directory Questions

Hello, need some ideas on the control statement for a script that will watch a directory, looking for any duplicate records as they are added.

I was thinking about a process that would watch the dir in a time window. Whenever a new line or entry was made I would grep the existing file for any exact lines that match.

Not sure about how to do this, maybe check on " ls | wc -l " or something similiar. Not sure about the control logic, while loop or a for statement.

Any ideas would be appreciated.

thanks, david

... just for clarification ...

  1. files come into directory
  2. files have records
  3. files may have duplicate records
  4. you want to remove duplicate records in files

right?