Read latest log files and perform database insert

Shell Programming and Scripting
1

Hi Experts,

I have a situation where I need to write a shell script to continuously monitor a log directory with multiple log files and perform following:

  1. Read the latest log file continuously and grep "Success" OR "Failure"
  2. As it capture either Success or Failure, it has to perform a database insert.
  3. Log files are regularly changing and rotating, so how can we pointing the current logfile seamlessly

For Eg.
Lets say the log directory is /var/log
Log Files:

app-log0
app-log1
app-log2
...

Now the shell script has to read the latest logfile " app-log2 " continuously and capture if "Success" OR "Failure" is coming. If it find it, perform the insert into a database table :

Insert into log_capture ...

And if a new log file created say " app-log3 ", it should start reading that log file and perform the same activity.

I know we can use tail -f that can continuously read a file, but I am not sure
1. how can I perform the database insert operation in between, whenever I get Success/Failure
2. How can " tail " pick a new file if the current file exhausted.

Any other suggestion/command is also welcome.

Thank you so much in advance.

2

That "reading continuously" doesn't sound feasible, and I don't see the advantage of doing so. Why don't you run a (set of) script periodically, check for old and new log files, and then insert a bunch of records into your DB?