Monitoring with Prometheus

Starting the demo

docker-compose up -d

Generating test load

docker run --rm -it --network=prometheus-demo nakabonne/ali ali -d 0 -r 50 http://demo-app:3000/slow

where -d is duration and -r rate (number of requests)

Sample Prometheus queries

App metrics

Observing Node.js event loop lag

Demo scenarios

Example 1

Run simultaneously:

docker run --rm -it --network=prometheus-demo nakabonne/ali ali -d 0 -r 150 http://demo-app:3000/fast
docker run --rm -it --network=prometheus-demo nakabonne/ali ali -d 0 -r 150 http://demo-app:3000/slow

Observe:

  • App metrics (rate and response time for two endpoints)

Example 2

Test a slow endpoint

docker run --rm -it --network=prometheus-demo nakabonne/ali ali -d 0 -r 250 http://demo-app:3000/slow

Test a slow endpoint that performs CPU intensive task on main thread

docker run --rm -it --network=prometheus-demo nakabonne/ali ali -d 0 -r 250 http://demo-app:3000/slow-blocking

Observe and compare:

  • App metrics (rate and response time for two endpoints)
  • Event loop lag
  • Possibly missing metrics (due to non-responsive /metrics endpoint)