Upgrading legacy packages with patch

Hi,

All of our production servers are installed with Apache and OpenSSL from source and not yum.
My boss has assigned me with a task of upgrading all production servers Apache httpd & openssl to the latest patch from Centos Repo.

Is it possible to do this via yum? Please suggest the best ways to perform this.

Thx.

Patching might not work - because the patch may assume versions of the link libraries that are different from what you linked in your original build.
Do you have a sandbox to test the patch? Try it there first. We have a couple of virtual boxes that we can play with for testing. We refresh them using the current production version. -- after we trash them.

I seriously doubt it. Installing from source and installing using a package manager (like YUM) are two different kettles of fish.

You are better off building the server from scratch in a test bed with your HTML files and testing first as Jim suggested.

Or you can also do as Jim suggested try upgrading using YUM on a TEST server (NOT production).

How many web servers are you talking about? 3? 10? 100?

We have about 200 - 300 production web servers.

1 Like

And how mission critical are these production servers?

What is the risk management profile of each one of them?

If they have a "high risk" profile if not available, the approach is different.

For example are all 300 web servers running the same application(s) and they are in a huge network of redundant web servers?

Or all all 300 web server running different applications, each with a different risk management profile?

Of all the 300 production web servers, most of them run different apps, and they are all divided into different segments, like 90, 130, 140, 80, 60.
Some (Very few) are used for load balancing.

Well, they all cannot have the same mission criticality and risk management profile.

Did you avoid answering these very important question for a particular reason?

All the servers are mission critical.
And in terms of risk management, we have an SLA of maximum 4 hours to bring the server back up in an event of a catastrophe.

Does not sound very "mission critical" to me.

If you define everything that can be down with an SLA for four hours as "mission critical", what would you define a server that if it went down it would cost the company 100K to 1M USD per hour?

Most people would not define a service as "MISSION CRITICAL" if it has a SLA of four hours, to be frank. But then again that depends on the "MISSION".

If you have SLA of four hours, then you can easily make a mistake and recover from it long before the four hour SLA window is reached. That is more like "A STANDARD BUSINESS SLA", for a lack of a better term.

Do you have a risk management team (normally a part of either the IT security or audit teams) responsible for the risk management of all these servers?

If so, get them involved.

The biggest loses any company has is usually a mistake by a well intended trusted employee. Often, these big mistakes are caused by trying to automate an upgrade to hundreds of devices (routers, servers, firewalls, etc).

Best to set up a test bed, work on the changes, and get it working. You cannot just take "YUM" and try to upgrade if the original installs were done manually. This is a formula for a lot of downtime!

Thank you for the clear and thorough information/explananation.
Yes, we do have IT Risk Management team. But I have no experience so far in getting them involved in any of the projects I have worked on.
What can they contribute if I get them involved?

Normally if we are making upgrades to mission critical applications in 100s of servers in a large organization with a risk management team we should notify them if if we plan any upgrades which may cause an outage.

You should test in a test bed, make a plan, identify the risks and notify your risk management team.

Upgrading the core "mission critical" application on 200-300 web servers requires planning and team work.

What happens if you make a mistake and bring down the application(s)? Do you want your organization to be caught off guard when customers are calling in, angry their service is down?

You have described a major infrastructure upgrade. You certainly do not want to upgrade with YUM over a manually installed configuration until you have completed tested this idea in a test bed, insure you have backups of each application and database before hand, etc. In other words, you need a plan in coordination with your IT risk management team, customer service, etc.

This is how we work as IT professionals.