runbook.md

runbook.md is a toolchain used for improving the quality of runbooks across the business, and ensuring consistency of the information contained within runbooks with biz-ops – the knowledge base which models our business operations.

Talk to us in #reliability-eng on Slack if you have any questions / suggestions.

Toolchain

Automated ingestion on publishing of release logs

All systems which integrate with Change API benefit from automated runbook.md ingestion and synchronisation with biz-ops, on every release that contains changes to a runbook.md file.

Please add Change API to your deployment pipeline and release your code. A successful journey will see any runbook.md file changes automatically applied in Biz Ops to the relevant system(s).

Here is the workflow:

GitHub app (beta)

The runbook.md github app validates a repository's RUNBOOK.MD files - Markdown files following the /runbook.md$/i naming convention - against our organisation-wide standard, and offers suggestions for improvement.

For manual release processes or multi-system repositories, where integrating with Change API is not possible, the GitHub app supports custom configuration (example here) for updating system information in Biz-Ops.

For repositories containing source code for more than one system, please adhere to a system-code_runbook.md naming convention (where system-code is the systemCode as written in Biz-Ops) or use custom config to map each runbook to a system.

Manual validation

This utility allows manual upload of a RUNBOOK.md file for validation and operability scoring. Optionally, the tool can be used to populate associated critical fields in Biz Ops based on the content of a valid runbook.

See an example of the format/content of a RUNBOOK.md file here.

System (business logic) overview

Installation and development

Prerequisites

Ensure Docker is installed and running
Ensure AWS CLI tooling is installed https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-install.html

Using GNU Make as a task runner:

make install installs dependencies and add linting config files
make env populates an .env file with the appropriate environment variables from Vault
make run runs the app which you can then visit at http://localhost:3000/runbook.md
make test runs tests
make verify runs linter checks (with eslint, using prettier)

Emulating Change API's release log message stream, offline

The task make run-local-message-stream sets up and starts an offline emulation of AWS' Kinesis. You can put records onto the resulting stream – change-request-api-test-enriched-stream – using the following AWS CLI command.

aws kinesis --endpoint-url http://localhost:4567 \
   put-record --stream-name change-request-api-test-enriched-stream \
   --partition-key “MyFirstMessage” \
   --data [stringifiedJSONreleaseLog](https://github.com/Financial-Times/runbook.md/blob/master/docs/change-api-example-message.json)

Troubleshooting common issues

The following error can occur when trying try to execute make run:

{
   errno: 'ENOTFOUND',
   code: 'ENOTFOUND',
   syscall: 'getaddrinfo',
   hostname: 'local.in.ft.com'
}

In this case, check your /etc/hosts file and add the entry: 127.0.0.1 local.in.ft.com.

How systemCode is picked

The number is the priority. It will fallback if the value is undefined.

Code from runbook.md
systemCodes From .github/runbooks.yml config (exapmle)
From file name ${system-code}_runbook.md
systemCode From Change API message [Change API route only]

obax/runbook.md