Apache Kafka starting in version 3.3 is using the Raft metadata protocol production ready to abandon the appendage ZooKeeper overhead to control the cluster's health.
More info can be found in the official Apache Kafka docs https://kafka.apache.org/documentation/#kraft
- added feature to configure kafka cluster with helm charts
- minor fixes
- added better ability to run the cluster dynamically in broker only mode
- redesigned debugmode
- Server mode (process.roles = broker, controller)
- Broker and Controller mode ((process.roles = broker) deploys a broker cluster and a controller cluster)
- Connect mode (100% dynamic)
- chart is available via Helm Repo -> https://stefanjay.github.io/kafka-kraft-on-k8s/
- Kafka KRaft on Kubernetes Helm Chart (K4C)
Stefan Jährling @ System Vertrieb Alexander GmbH
✨ As several repositories on github share similar approaches, I added my own. Inspired by some of them for you to try out. ✨
- Helm Chart (feat: RBAC, SA, HPA, AS)
- featuring StatefulSets
- clustered with at least 3 nodes
- Kafka logdata on persistent volume /node
- environment variables in manifest
- up-to-date light-weight Kafka Connect Cluster
- add connectors as easy as build an image
- runs in any containerized environment
- added feature to configure kafka cluster with helm charts
- minor fixes
- added better ability to run the cluster dynamically in broker only mode
- redesigned debugmode
- fixed Kafka Connect to be fully dynamic
- added KRaft controller
- added Broker process.roles
- added several parameters to values.yaml for flexibility
- Minor fix (package version, image name in values.yaml)
- Refactored repository structure
- First Helm release
- JMX definition
- Kafka Connect Splunk integration
- Kafka Connect
- Minor manifest changes
- Replication factor
- Added topic autocreation env
- Bug: Kafka runs as nonroot user in some cases
- Bumped Kafka Connect to v3.6.0
- Bumped base image to openjdk.22-bookworm
- Open fixme for DEFAULT_REPLICATION_FACTOR
- Bumped Kafka to v3.6.0
- Switched back to root user for compatibility
- Backported to Kafka v3.3.2 due to unfixed bug
- Kafka Connect
- Updated GitHub Actions
- Bumped Kafka to v3.4.0
- Bugfixed nonroot app-dir to /opt/kafka
- Kafka data logdir in /mnt/kafka as PV
- CLUSTER_ID can be set in manifest as ENV
- Bugfix: Kafka random bootstrap error due to empty cluster_id
- Cluster_id is hard coded until fix
- Separate PVCs (100Gi) for each replica
- Unprivileged user to run as non-root
- Using Kafka v3.3.2
- Dockerfile
- Updated entrypoint.sh
- Switched to Kafka v3.3.1
