Web Site Performance Issue

Hi All,

I am currently managing a web site. I find that during the peak time period,ie,between 1300hrs to 2100hrs everyday,the response slows down whenever i access the applications hosted by the server.However when i try to access the static content in the web server,it loads quickly.
The architecture is as below:

client --> webserver --> appserver --> dbservers.

OS - RHEl 5.4
Web Server - Apache 2.2.23
App Server - Jboss 4.2.3

While debugging i found that when the Thread count reaches 260 in webserver then the issue starts. However there are days when i have seen no issue even if the thread count is 260.Also even if thread count has reached 260,http requests for static content requests are processes without any delay.Only the "https" requests that access the application are not responding properly.

I have searched the thread count in Appserver and it is below "200" and the Max count has been set to "250". So i am not sure how to debug further. Any help is greatly appreciated.

BTW,i am talking about a domain which hardly receives around 3000 hits a day(as per Awstats report). So it really perplexes me.

Thanks
HG

Read up on sar. Here's an example.

Here's another solution: between 1300 and 2100 run the ps command every minute. Append the output to a file which is rotated. Look at the file every day.

sid=`ps --pid $$ --no-headers -o sid`
ps -N --sid $sid -C sleep --sort start_time,comm,pid -ww \
         -o pid,c,pmem:5,rss:8,sz:8,size:8=TSIZE,vsz:8,nlwp,wchan:16,start_time,tty:8,args

You could make ascript out of it ("ps-snapshot" for instance) and add it to a cron file to run every few minutes like this:

*/3 * * * * /usr/local/bin/ps-snapshot >> /var/log/ps

Make sure you add /var/log/ps to the list of files to be rotated in /etc/logrotate.d

Thanks Otheus for the response.
I had the server stats collected through Nmon and also through my monitoring utilities and didn't see much difference.
Anyway i will implement your script and see if anything turns up.

Thanks
HG