Kafka Performance Tester
Introduction
Kafka Performance Tester is a tool to load test your Kafka cluster. It uses Apache Kafka's inbuilt kafka-producer-perf-test.sh
and kafka-consumer-perf-test.sh
utilities and additional tools to make the test suit ready to roll.
Requirements
- Docker
- A running Kafka cluster(With one or multiple brokers)
Supported Platforms
Kafka Performance Tester can be used on any platform, thanks to Docker! Currently the standard configuration is created for DC/OS only which resides in options.json
. In coming days, a standard deployable config for Kubernetes will also be added.
Quick Start
Set the necessary environment variables.
- For DC/OS, set the parameters in
options.json
. - For Kubernetes, use
deployment.yaml
- For local system, set the vars manually inside the docker container using:
-
docker run -it rishabh96b/kafka-workload:latest bash export NUM_TOPICS=30 export NUM_RECORDS=1024 export RECORD_SIZE=50 export PRODUCER_THROUGHPUT=10 export TEST_INTERVAL_SECONDS=5 export BOOTSTRAP_SERVERS="http://localhost:9092"
-
To run Producer benchmarks:
./producer_test.sh
To run Consumer benchmarks:
./consumer_test.sh