🛑 Xibo Cloud United Kingdom/London is down

Question

🛑 Xibo Cloud United Kingdom/London is down

Closed this issue a year ago · 5 comments

In ec5076a, Xibo Cloud United Kingdom/London ($CLOUD_GB_URL) was down:

HTTP code: 0
Response time: 0 ms

Answer 1 · 2023-08-09T11:09:04.000Z

Investigating: This is a continuation of the earlier incident. There is a networking outage with our data center partner. We are working with them to resolve the problem.

Answer 2 · 2023-08-09T11:15:23.000Z

Resolved: Xibo Cloud United Kingdom/London is back up in 054e365.

Answer 3 · 2023-08-09T11:17:57.000Z

Monitoring: Service has been restored again. We await confirmation that no further disruption is expected.

Answer 4 · 2023-08-09T16:18:33.000Z

Resolved: We are advised that the issue has been resolved and that no further interruption is expected. Apologies for the inconvenience we know this has caused.

Answer 5 · 2023-08-14T12:05:52.000Z

Postmortem: DigitalOcean have provided a postmortem of the incident. The issue was related to a router firmware bug. They are taking steps to ensure that a similar issue cannot cause the same kind of incident again, including migrating to a multi-vendor routing setup such that a similar bug is unlikely to impact primary and fail-over routers concurrently.