Any utility or tool to check logs

Shell Programming and Scripting
1

HI,
In our project daily we need to check for some errors in around 45-50 folders. Please let me know if there is any utility tool using which 1 can check each folder and file for error.
Use: Monday we are checking if there were any error in files generated on Monday
Tuesday to Wednesday we need to check if there are any files created on previous day with the error.
Please let me know if there is any tool or utility in unix to check this that we can run daily to extract the error data

2

Hi,

What generates the errors and how do you check the files/folders just now?

Regards

Dave

3

Hi Dave,

our process are running and they are developed in such manner that they create errorfile.txt in error folder of that process.
I can write shell script to monitor each and every folder.but wanted to know if there is any tool using which we can check this instead of writing script.

Please let me know if my requirement is not clear

4

Hi Ankush,

It seems that there may be a requirement to write your own scripts here, in general all my scripts log to a single error log, which is generated one per day so it is easy for me to see what failed and when.

But if each "process" generates an "errorfile.txt" in a different location, then you will I fear have to write a specifi script to check for that.

Regards

Dave

5

Thanks Dave.
Will write script... there is no utility or tool using which we can check that.

6

grep -c regex folder ?

7

Hi,

As I said, I have a single log file which all my scripts log to. So I don't have this issue, perhaps rather than write a script that would have to be continually edited - you may want to look at implementing a more considered methodology.

Regards

Dave

8

Welcome ankush_mehra ,

Can you tell us more about the directory stricture that the log files will be written to, eg:-

  • Are they /logs/Monday , /logs/Tuesday etc. or /logs/2014-09-09 style. If so, what format?
  • Are the logs single files for the day or one for each job (named for the job, process id, timestamp etc.) ?
  • Are the contents of the log files (however organised) consistent so you can search for a particular string or expression?
  • Are the log files archived, overwritten or just forever growing in number?

A good response also depends on what you have tried so far and what you are comfortable with. It would be better if we can work out what you know so that you can maintain any solution, rather than us just giving something that works in a test situation, but then is hard for you to adapt for actively using it.

It would help us all if you could consider these questions:-

  • What have you tried so far?
  • What output/errors do you get and how far off are you?
  • Can you supply some sample input and output required? (wrapped in CODE tags please)
  • What OS and version are you using?
  • What are your preferred tools? (C, shell, perl, awk, etc.)
  • What logical process have you considered? (to help steer us to follow what you are trying to achieve)

Most importantly, What have you tried so far?

There are probably many ways to achieve most tasks, so giving us an idea of your style and thoughts will help us guide you to an answer most suitable to you so you can adjust it to suit your needs in future.

We're all here to learn and getting the relevant information will help us all.

Kind regards,
Robin