KafkaVision simplifies monitoring of a Kafka cluster by allowing developers to:
- Quickly view a snapshot of a cluster's health
- Visualize and compare topic level metrics in a cluster
How KafkaVision saves developers time:
- Prioritizes cluster health metrics and status updates that really matter
- Provides relevant and useful graphs that update in real-time -- with no additional configuration required
- Organizes topics and partitions in an easy-to-consume visualization to inform quick consumer optimization decisions
- Eliminates the need for manual creation of PromQL queries and unfamiliar CLI commands
To use this application, you'll need to have at least one of the following:
- A running Kafka instance with at least one broker with a public port
- A running Kafka instance configured with Prometheus and access to the corresponding Prometheus port
KafkaVision enables you to view your cluster information in the following ways using an interactive GUI:
1) View Cluster Health Metrics (with Prometheus Connection)
Users can view the overall health of their cluster at a glance on the 'Health Metrics' page which includes auto-updating graphs displaying real-time data readings for CPU Usage, JVM Memory Used, and Time spent in GC. It also includes static metrics at the bottom of the page for counts of key cluster elements:
2) View Data Throughput at Topic-Level for TOP 5 High-Use Topics (with Prometheus Connection)
The 'Topic Metrics' page displays real-time throughput metrics for the most active Cluster topics. Click on each topic section to expand and view graphs displaying throughput information at the topic level and click again to collapse.
3) Insights into cluster topics, partitions, and consumer offsets (with Broker connection)
The 'Partition Diagrams' page allows users to gain insight into the partitions within each Topic of their Cluster, including the number of partitions and the current offsets, allowing developers to quickly make optimization decisions about partition and consumer assignments.
- Clone this repo on your local machine
- Install dependencies with
npm install - Compile Typescript with
npm run compile-prod - In a new terminal, run the command
npm run build-prod - Then, run
npm start - Open localhost:3333 to enter the KafkaVision App
- To add a new broker or Prometheus address, add the location in the input field and click Submit.
If you do not have a Kafka instance but would like to demo KafkaVision, we modified this public demo app from Confluent Platform to scrape metrics from the cluster and export them to Prometheus using JMX-Exporter. To quickly get up and running, you can open our pre-configured demo cluster with GitPod and follow the boot instructions.
The development team intends to continue improving KafkaVision and adding more features. Head to our roadmap to see our upcoming planned features.
- TypeScript - Programming Language
- Node.js - File system, testing, core application functionality
- Express - Backend Framework
- Webpack 5 - Module Bundler
- React - Frontend Framework
- KafkaJS - Kafka client for Node.js
- Prometheus - Time series database with event monitoring
- ChartJS - Rendering of metric graphs
- Tailwind CSS - Styling components
- Jest - Testing
Isabelle Salvador - LinkedIn
Kayliegh Hill - LinkedIn
Neel Lakshman - LinkedIn
Rob Beier - LinkedIn
If you'd like to support the active development of KafkaVision:
- Star this repo!
- Tweet about the project
- Write a review or tutorial on Medium, Dev.to or personal blog
- Contribute to this project by raising a new issue or making a PR to solve an issue
This project is licensed under the MIT License - see the LICENSE file for details