ministryofjustice/serve-opg

Serve OPG PagerDuty configuration

Closed this issue · 1 comments

In relation to #225 we need to configure pager duty for all our alerts.

I have already started to do this, although I would like to redo build it using Terraform so we can manage it as code. (It's not very hard to do this and I have already made a start in my own time).

At the moment I am planning to implement an email in which incidents can be raised from and also integrate it with our cloud watch alarms.

Reasons for Pager Duty:

  • Allow everyone in the team to receive alerts if they want
  • Allows team members to choose how they receive alerts
  • Centralise all Alerts into one place
  • The easiest way to comply with the new guidance from central MOJ in regards to providing a contact point for the team.
  • Incidents can be addressed and monitored.
  • If we have a major outage, PD allows us to conduct a postmortem of the issue
  • Allows other opg product teams to raise incidents that we might need to know about when we start the next day.
  • If we ever need to move to a model of providing out of hours support having PD configured would allow us to do this in minutes
  • The above would all be managed through code in opg-org-infra