kevin-Murphy-hi/flink-prometheus-example

Example setup to demonstrate Prometheus integration of Apache Flink

This repository contains the live demo to my talk Monitoring Flink with Prometheus, which I have given at:

• Flink Forward Berlin 2018, 2018-09-04 (:video_camera: Video 📄 Slides)
• Spark & Hadoop User Group Munich, 2018-09-26

The blog post Flink and Prometheus: Cloud-native monitoring of streaming applications explains how to run the demo yourself.

Getting Started

Startup

./gradlew composeUp

Web UIs

(When using docker-machine, substitute your docker-machine ip for localhost in the URLs.)

• Flink JobManager
• Prometheus
• Grafana (credentials admin:flink)
• Prometheus endpoints
  • Job Manager
  • Task Manager 1
  • Task Manager 2

Built With

• Apache Flink
• Prometheus
• Grafana
• docker-compose – provisioning of the test environment
• Gradle with kotlin-dsl
  • gradle-testsets-plugin
  • shadow
  • spotless
  • spotbugs
  • gradle-docker-compose-plugin
  • gradle-versions-plugin

Development

typical tasks:

• verify: ./gradlew check
• integration tests: ./gradlew integrationTest
• list outdated dependenices: ./gradlew dependencyUpdates
• update gradle: ./gradlew wrapper --gradle-version=<x.y> (twice)