This repo simulates monitoring HTTP microservices and short-lived batched jobs with Prometheus.
Grafana is deployed alongside Prometheus with sample dashboards for visualizing the data.
Before running any of the demonstrations below, you need to build the docker image for running the test kv_store and aggregator applications.
make imageTo start the servers, use docker-compose.
docker-compose upYou will have to wait a few minutes before metrics start to appear on the dashboards described in the following sections.
The docker-compose.yaml configuration sets up three instances of an HTTP key/value store microservice (similar to memcached).
- The code that implements of the key/value store with prometheus instrumentation in
kv_store.py. - The code that synthesizes traffic on these instances can be found in
simulate_http_traffic.py.
The Grafana server is preconfigured with the dashboard for HTTP metrics shown below.
The docker-compose.yaml configuration sets up a metrics aggregator that short-lived jobs can send their metrics to.
- The code that implements the metrics aggregator lives in
aggregator.py. - The code that synthesizes batch job metrics is in
simulate_jobs.py
The Grafana server is preconfigured with the dashboard job metrics shown below.
You will then have the following servers running.
You can check the metrics collection status of the kv_store servers on the Prometheus targets page.
On the grafana server there is a sample HTTP dashboard showing off some standard http metrics.
Shut it down when you're done!
docker-compose down