Montioring Script Tomcat catalina.out

Hi,

I need to monitor the tomcat log file called "Catalina.out" for any errors like out of memory,JDBC exceptions,broken pipe any other errors and exceptions

The script should monitor this file and send a mail instantly and create a text file with errors day to day.

Iam using Solaris 10.Iam new to this.

can you suggest me which is the best way to do this?

What have you tried to do so solve this problem on your own?

What is the format of the data placed in Catalina.out by tomcat for:

  • an out of memory error,
  • a JDBC exception,
  • a broken pipe,
  • any other errors you want to catch, and
  • any other exceptions you want to catch?

In what directory is Catalina.out located?

Who is the owner and what is the group ID of Cataline.out ?

What are the permissions on Cataline.out ?

What are the effective and real user and group IDs of the script that will be processing Catalina.out ?

Is any data written by tomcat into Catalina.out that you do not want to trigger immediate e-mail?

To whom should this e-mail be sent?

What mail sending utility should be used to send e-mail? (Maybe mail , mailx , or sendmail ?)

What text to you want to be included in these e-mails?

What shell are you using?

Is Catalina.out kept open by tomcat on your system during time periods when it is not actively composing a message to be logged?

What is the name of the text file(s) that should be created day to day?

What exactly should be written into the day to day text file(s)?

One of the more important things of Tomcat Monitoring is to keep an eye on the memory usage. The solution for most Tomcat/Java Server Problems is...

More Memory!

Some say there's only one better thing than more memory for a Tomcat Server:

Even more Memory!

And for the monitoring part: I suggest you to have a more broader look at your problem. When you have a certain message "...bla...broken pipe...", then you may know it already "Ah this one again" or you dig into your system until you figure out the root cause of the problem. What helps a lot in discovering the reasons are system performance data.

  • Tomcat Java Memory Usage over time
  • System RAM Usage over time
  • IO-Usage and Filesystem growth over time
  • CPU-Usage
  • Load
  • Network(Bandwidth+Errors)
  • Clock-synchronisation and drift(Very important!)

If you have those data, you'll discover from a look at those graphs what is normal for your system at what is not and above all: you'll discover all sort of changes before a critical state is reached. You'll learn what the impact of certain technical changes is, when you see the data coming in after that.

I suggest you to have a look at some monitoring system. As I assume you work in some type of company, there's probably already a monitoring system installed. I would expect most monitoring systems to be flexible enough to add your specific own checks there or use the features already available there. If no monitoring system is available to use, I recommend monit as an easy one ore check_mk as a more powerful approach - the latter needs to be on a separate best dedicated system(vps is enough). (Maybe there's a better solution for solaris available, but I do not know that).

There are some checks available for Tomcat/Java and check_mk. The client(=your tomcat server) part consists of a bash script most of the times. Sometimes, so chances are good that they just work at your server. Look here: Catalog of check plugins

And of course that one you're after(log file pattern matching) may be an import issue too. Most monitoring applications do have such features available.

Thanks Don,

My log looks like this.

Sep 11, 2016 10:19:02 AM org.apache.catalina.core.ApplicationContext log
INFO: Initializing Spring root WebApplicationContext
log4j:WARN No appenders could be found for logger (org.springframework.web.context.ContextLoader).
log4j:WARN Please initialize the log4j system properly.
[GC 4532642K->390826K(16080384K), 0.2922149 secs]
Sep 11, 2016 10:19:28 AM org.hibernate.cache.SingletonEhCacheProvider buildCache
WARNING: Could not find a specific ehcache configuration for cache named [com.clarity.core.security.iface.access.domain.Privilege]; using defaults.
Sep 11, 2016 10:19:29 AM org.hibernate.cache.SingletonEhCacheProvider buildCache
WARNING: Could not find a specific ehcache configuration for cache named [org.hibernate.cache.UpdateTimestampsCache]; using defaults.
Sep 11, 2016 10:19:29 AM org.hibernate.cache.SingletonEhCacheProvider buildCache
WARNING: Could not find a specific ehcache configuration for cache named [org.hibernate.cache.StandardQueryCache]; using defaults.
Sep 11, 2016 10:19:32 AM org.apache.catalina.core.StandardContext listenerStart
SEVERE: Exception sending context initialized event to listener instance of class com.clarity.webservices.spring.ClarityContextLoaderListener

Just want to get the errors or exceptions in Log for montioring. The application is deployed in tomcat user and has all permission. Need to send mail with the error that occurred. Any sample code ?