A lightweight Go Web Server that receives POST alert messages from Prometheus Alert Manager and sends it to a Microsoft Teams Channel using an incoming webhook url. How light? The docker image is just 7 MB!
Alertmanager doesn't support sending to Microsoft Teams out of the box. Fortunately, they allow you to use a generic webhook_config for cases like this. This project was inspired from idealista's prom2teams which was written in Python.
Why use Go? A Go binary is statically compiled unlike the other simple language (python, ruby, node). Having a static binary means that there is no need for you to install your program's dependencies and these dependencies takes up a lot of space in your docker image! Try it out DevOps folks!
- Getting Started (Quickstart)
- Sending Alerts to Multiple Teams Channel
- Customise Messages to MS Teams
- Kubernetes Deployment
- Developers Build Guide
How it works.
OPTION 1: Run using docker.
docker run -d -p 2000:2000 \
--name="promteams" \
-e TEAMS_INCOMING_WEBHOOK_URL="https://outlook.office.com/webhook/xxx" \
-e TEAMS_REQUEST_URI=alertmanager \
docker.io/bzon/prometheus-msteams:v1.1.4OPTION 2: Run using binary.
Download the binary for your platform from RELEASES, and run it like the following:
./prometheus-msteams server \
-l localhost \
-p 2000 \
-w "https://outlook.office.com/webhook/xxx"OPTION 3: If you are going to deploy this in a Kubernetes cluster, checkout the Kubernetes Deployment Guide.
By default, prometheus-msteams creates a request uri handler /alertmanager.
route:
group_by: ['alertname']
group_interval: 30s
repeat_interval: 30s
group_wait: 30s
receiver: 'prometheus-msteams'
receivers:
- name: 'prometheus-msteams'
webhook_configs: # https://prometheus.io/docs/alerting/configuration/#webhook_config
- send_resolved: true
url: 'http://localhost:2000/alertmanager' # the prometheus-msteams proxyIf you don't have Prometheus running yet and you wan't to try how this works,
try stefanprodan's Prometheus in Docker to help you install a local Prometheus setup quickly in a single machine.
Create the following json data as prom-alert.json.
{
"version": "4",
"groupKey": "{}:{alertname=\"high_memory_load\"}",
"status": "firing",
"receiver": "teams_proxy",
"groupLabels": {
"alertname": "high_memory_load"
},
"commonLabels": {
"alertname": "high_memory_load",
"monitor": "master",
"severity": "warning"
},
"commonAnnotations": {
"summary": "Server High Memory usage"
},
"externalURL": "http://docker.for.mac.host.internal:9093",
"alerts": [
{
"labels": {
"alertname": "high_memory_load",
"instance": "10.80.40.11:9100",
"job": "docker_nodes",
"monitor": "master",
"severity": "warning"
},
"annotations": {
"description": "10.80.40.11 reported high memory usage with 23.28%.",
"summary": "Server High Memory usage"
},
"startsAt": "2018-03-07T06:33:21.873077559-05:00",
"endsAt": "0001-01-01T00:00:00Z"
}
]
}curl -X POST -d @prom-alert.json http://localhost:2000/alertmanagerThe teams channel should received a message.
You can configure this application to serve 2 or more request path and each path can use a unique Teams channel webhook url to post.
This can be achieved by supplying the application a configuration file.
Create a yaml file with the following format.
connectors:
- high_priority_channel: "https://outlook.office.com/webhook/xxxx/aaa/bbb"
- low_priority_channel: "https://outlook.office.com/webhook/xxxx/aaa/ccc"NOTE: high_priority_channel and low_priority_channel are example handler names.
When running as a docker container, mount the config file in the container and set the CONFIG_FILE environment variable.
docker run -d -p 2000:2000 \
--name="promteams" \
-v /tmp/config.yml:/tmp/config.yml \
-e CONFIG_FILE="/tmp/config.yml" \
docker.io/bzon/prometheus-msteams:v1.1.4When running as a binary, use the --config flag.
./prometheus-msteams server \
-l localhost \
-p 2000 \
--config /tmp/config.ymlThis will create the request uri handlers /high_priority_channel and /low_priority_channel.
To validate your configuration, see the /config endpoint of the application.
curl localhost:2000/config
[
{
"high_priority_channel": "https://outlook.office.com/webhook/xxxx/aaa/bbb"
},
{
"low_priority_channel": "https://outlook.office.com/webhook/xxxx/aaa/ccc"
}
]Considering the prometheus-msteams config file settings, your Alert Manager would have a configuration like the following.
route:
group_by: ['alertname']
group_interval: 30s
repeat_interval: 30s
group_wait: 30s
receiver: 'low_priority_receiver' # default/fallback request handler
routes:
- receiver: high_priority_receiver
match:
severity: critical
- receiver: low_priority_receiver
match:
severity: warning
receivers:
- name: 'high_priority_receiver'
webhook_configs:
- send_resolved: true
url: 'http://localhost:2000/high_priority_channel' # request handler 1
- name: 'low_priority_receiver'
webhook_configs:
- send_resolved: true
url: 'http://localhost:2000/low_priority_channel' # request handler 2This application uses a default Microsoft Teams Message card template to convert incoming Prometheus alerts to teams message cards. This template can be customised. Simply create a new file that you want to use as your custom template. It uses the Go Templating Engine and especially the Prometheus Alertmanager Notification Template. Also see the Office 365 Connector Card Reference and some examples for more information to construct your template. Apart from that, you can use the Message Card Playground to form the basic structure of your card.
When running as a docker container, mount the template file in the container and set the TEMPLATE_FILE environment variable.
docker run -d -p 2000:2000 \
--name="promteams" \
-e TEAMS_INCOMING_WEBHOOK_URL="https://outlook.office.com/webhook/xxx" \
-v /tmp/card.tmpl:/tmp/card.tmpl \
-e TEMPLATE_FILE="/tmp/card.tmpl" \
docker.io/bzon/prometheus-msteams:v1.1.4When running as a binary, use the --template-file flag.
./prometheus-msteams server \
-l localhost \
-p 2000 \
--template-file /tmp/card.tmplSee Helm Guide.
Build
make dep
make darwin # for osx
make linux # for linuxTest
make testDocker Build
make docker VERSION=v1
make docker-push VERSION=v1