Kafka MirrorMaker Helm Chart
Helm Chart for Production Grade Kafka MirrorMaker deployment on Kubernetes inspired by Confluent Helm charts (https://github.com/confluentinc/cp-helm-charts )
Kubectl
GKE Cluster (Kubernetes 1.9.2+)
Helm 2.8.2+
A healthy and accessible Kafka Cluster
// create cluster
gcloud container clusters create test
gcloud container clusters get-credentials test
//initialise helm
helm init
// override helm issue with rbac
kubectl create serviceaccount --namespace kube-system tiller
kubectl create clusterrolebinding tiller-cluster-rule --clusterrole=cluster-admin --serviceaccount=kube-system:tiller
kubectl patch deploy --namespace kube-system tiller-deploy -p '{"spec":{"template":{"spec":{"serviceAccount":"tiller"}}}}'
//deploy mirrormaker
helm install --set kafka.consumerConfig.bootstrap\.servers="dev3-kafka:9092",kafka.producerConfig.bootstrap\.servers="dev4-kafka:9092" ./og-kafka-mm/
(Also included in ./docker-src
)
Kafka MirrorMaker Deployment
The configuration parameters in this section control the resources requested and utilized by the og-kafka-mm
chart.
Parameter
Description
Default
replicaCount
The number of Kafka Kafka MirrorMaker Servers.
1
Parameter
Description
Default
image
Docker Image of Kafka MirrorMaker.
opsguruhub/og-kafka-mm
imageTag
Docker Image Tag of Kafka MirrorMaker.
latest
imagePullPolicy
Docker Image Tag of Kafka MirrorMaker.
IfNotPresent
imagePullSecrets
Secrets to be used for private registries.
see values.yaml for details
Kafka MM JVM Heap Options
Parameter
Description
Default
heapOptions
The JVM Heap Options for Kafka MirrorMaker
"-Xms1024M -Xmx1024M"
Parameter
Description
Default
resources.requests.cpu
The amount of CPU to request.
see values.yaml for details
resources.requests.memory
The amount of memory to request.
see values.yaml for details
resources.requests.limit
The upper limit CPU usage for a Kafka Connect Pod.
see values.yaml for details
resources.requests.limit
The upper limit memory usage for a Kafka Connect Pod.
see values.yaml for details
Parameter
Description
Default
podAnnotations
Map of custom annotations to attach to the pod spec.
{}
Parameter
Description
Default
jmx.port
The jmx port which JMX style metrics are exposed.
9998
jmx.enabled
Whether or not to install Prometheus JMX Exporter as a sidecar container and expose JMX metrics to Prometheus.
true
Prometheus JMX Exporter Configuration
Parameter
Description
Default
prometheus.jmx.image
Docker Image for Prometheus JMX Exporter container.
solsson/kafka-prometheus-jmx-exporter@sha256
prometheus.jmx.imageTag
Docker Image Tag for Prometheus JMX Exporter container.
6f82e2b0464f50da8104acd7363fb9b995001ddff77d248379f8788e78946143
prometheus.jmx.port
JMX Exporter Port which exposes metrics in Prometheus format for scraping.
5556
prometheus.jmx.resources
JMX Exporter resources configuration.
see values.yaml for details
Parameter
Description
Default
nodeSelector
Dictionary containing key-value-pairs to match labels on nodes. When defined pods will only be scheduled on nodes, that have each of the indicated key-value pairs as labels. Further information can be found in the Kubernetes documentation
{}
tolerations
Array containing taint references. When defined, pods can run on nodes, which would otherwise deny scheduling. Further information can be found in the Kubernetes documentation
{}
Parameter
Description
Default
kafka.topicWhitelist
Regex for a topic whitelist
".*"
kafka.numStreams
Number of consumers to create
"1"
kafka.overrideGroupId
Specify custom group.id. Do not specify group.id in kafka.consumerConfig
- use this variable to override the default
".Release.Name"
kafka.consumerProperties
Kafka consumer settings. This should include kafka.bootstrap.servers
kafka.producerProperties
Kafka producer settings. This should include kafka.bootstrap.servers