There and Back Again: Community Site Down and Back Up Due to a Docker Issue

Today around 1AM New York time, our site went down due to a docker networking issue. We could not easily fix this issue, so at 4:30 AM NY time, I decided to move the entire site to a different host; which was completed by 6:30 AM NY time.

The problem with this move was that all the many thousand images needed to be re-optimized and that process was ongoing.

Then, I found that the issue was related to internet DNS inside the docker container on the original host; and while trouble shooting (after the fact), I fixed the issue on the original host by restarting docker.

So, I decided to move back to the original host where all the images are optimized (since we fixed the docker failure by restarting docker).

This was the first time in two years working with docker that I've had to restart docker to clear a problem.

It's been a long day!

Thank you for your patience.

7 Likes