Help needed in fixing port issue - Urgent

Linux and Unix-Like Red Hat
1

Hi,
One of our web application is running in Linux and using apache web servers. Application is hosted in port 4080 (for soap requests) and port 9999, and its serving behind a vip and cname. Initially it was working fine in another server and recently (2 months back) we have moved to the new server and from there onwards often ports 4080 and 9999 are failing.
As per our analysis there is no problem in the load balancer where the vip is served and also we didn't see any issues in the apache/application logs. But after checking with networking teams we found that at the time of port failures, health check (status.html) for the vip is failing and that causing the issue. Sometimes within 5 mins ports are coming up automatically without taking any actions and sometimes its not coming up and we are restarting the apache services.
We could not able to figure out whats the problem here, could you please someone help me how I can find the root cause of the issue.
Its a P1 production issue, please help me on this.

Thanks.

2

How Do you perform the vip fail-over? Is the ARP cache correctly updated on the host running the health check after the fail-over took place?

3

Hi,
I hope so, because vip part is totally taken care by another team and we had multiple discussions with them on this issue, but as per them there are no errors in the load balancer (from where vip is served) logs and they able to see only the errors which are related to health check failure. They are telling while lb could not able to do the health check bcoz of our port issue.
Could you please tell me how i can check on this issue.

Thanks.

4

What "port issue" is that? (Connection refused?)

Without knowing your particular setup, I would test as follows:

  1. check if telnet on ports 4080 and 9999 is OK,
  2. trigger failover
  3. ping the vip [to eventually update my ARP cache]
  4. check again if telnet on ports 4080 and 9999 is OK

Then, I'd check if it makes a difference when you drop step 3.

HTH,
Lo�c

5

Hi,
Thanks for your suggestions, will check and let you know on this.

Regards,
Ram