Meta is Down

Neo · August 15, 2020, 2:40am

# Oops

The software powering this discussion forum encountered an unexpected problem. We apologize for the inconvenience.

Detailed information about the error was logged, and an automatic notification generated. We'll take a look at it.

No further action is necessary. However, if the error condition persists, you can provide additional detail, including steps to reproduce the error, by posting a discussion topic in the site's feedback category.

Neo · August 15, 2020, 2:44am

Currently down for 15 minutes +

Neo · August 15, 2020, 2:48am

20+ minutes

DiscourseDuck · August 15, 2020, 2:53am

Seems meta is back up with total down time of a bit over 20 minutes, with my theme choice gone:

Neo · August 15, 2020, 2:56am

Changed to dark theme on mete and got this:

Neo · August 15, 2020, 2:57am

Now OK

Close to 30 mins down time total.

Neo · August 15, 2020, 9:35am

My recommendation is everyone who runs Discourse run it behind a reverse proxy and only rebuild a container which is "not live" and when the standby container is rebuilt, switch it to "live" and if there is a problem, switch back.

Let's call the current running container: LIVE
Let's call the current standby container: STANDBY

When you want to rebuild, rebuild STANDBY. When STANDBY is finished building, wait at least one minute and then switch over to it via your reverse proxy configuration. Reload Discourse in the browser and test. If all goes well, you are "good to go" with zero downtime.

STANDBY become LIVE
LIVE become STANDBY

If there is a problem after you switch STANDBY to LIVE, you can always switch back; with minimal downtime.

The only exception to this is when the container being rebuilt is the data container, versus the app container(s). However, there are also good ways to mitigate downtime if you ever have to rebuild the data container.