Add an existing kafka cluster to CMAK
Closed this issue · 4 comments
Hi I have an existing Kafka cluster with ZooKeeper cluster in EKS in AWS. (I built this cluster by using Strimzi operator and Kafka Kind). I failed to add this existing cluster to CMAK, after I successfully installed CMAK by cmak-operator chart. Is there anything wrong with my procedure?
kubectl -n kafka get pods
NAME READY STATUS RESTARTS AGE
cn-northwest-1-ops-kafka-crd-entity-operator-546c96d747-nggbz 3/3 Running 0 38s
cn-northwest-1-ops-kafka-crd-kafka-0 1/1 Running 0 2m41s
cn-northwest-1-ops-kafka-crd-kafka-1 1/1 Running 0 2m41s
cn-northwest-1-ops-kafka-crd-kafka-2 1/1 Running 0 2m41s
cn-northwest-1-ops-kafka-crd-zookeeper-0 1/1 Running 3 (3m44s ago) 5m45s
cn-northwest-1-ops-kafka-crd-zookeeper-1 1/1 Running 4 (3m4s ago) 5m45s
cn-northwest-1-ops-kafka-crd-zookeeper-2 1/1 Running 0 5m45s
strimzi-cluster-operator-5c8c984b7b-tcfnt 1/1 Running 0 4h22m
From the above screenshot, I added zookeeper hosts info into the second box.
cn-northwest-1-ops-kafka-crd-zookeeper-0:2181, cn-northwest-1-ops-kafka-crd-zookeeper-1:2181, cn-northwest-1-ops-kafka-crd-zookeeper-2:2181/kafka
Then, got timeout error:
kubectl -n cmak get pods
NAME READY STATUS RESTARTS AGE
cmak-76cffb897f-jfvxz 2/2 Running 0 138m
cmak-reconcile-27547128--1-89kgd 0/1 Completed 0 6m34s
cmak-reconcile-27547131--1-6scp6 0/1 Completed 0 3m34s
cmak-reconcile-27547134--1-dlgrg 0/1 Completed 0 34s
log from the pod, cmak-reconcile-xxxxxxx
2022-05-17 22:54:01,299 Connecting to cmak.cmak(172.20.198.187):2181, use_ssl: False
2022-05-17 22:54:01,303 Zookeeper connection established, state: CONNECTED
2022-05-17 22:54:01,305 Closing connection to cmak.cmak:2181
2022-05-17 22:54:01,305 Zookeeper session closed, state: CLOSED
- Readme for adding kafka clusters to cmak https://github.com/eshepelyuk/cmak-operator#kafka-clusters
- are cmak and kafka/zk in the same kind ckuster, same namespaces ?
- are used address accesible from cmak container, i.e. calling nslookup and/or telnet
1: I read the link you provided, not sure if I understand correctly. My situation is that I already have a Kafka cluster created by Strimzi Operator. Then I want to use CMAK to manage this cluster visually. I do not need to use CMAK to build a new cluster. Is this correct? (Not sure if CMAK can build new kafka/zk cluster?) Based on your doc, looks like that you are using CMAK to create a new Kafka/zk cluster. Can you explain a little more here?
2: CMAK and my Kafka/zk are not in the same namespace. (But they are in the same EKS cluster.) CMAK is in the namespace, cmak, my kafka/zk cluster is in: kafka
3: About "address", do you mean Cluster Zookeeper Hosts? About Cluster Zookeeper Hosts, is this EC2 hostname, where ZK pod sit?
-
neither cmak nor cmak-operator docs never mentioned that cmak can be used to create kafka cluster. have absolutely no idea where did you pick up that thought from.
-
check those ZK addresses from within cmak container with nslookup and telnet. the tools are not installed in
ui
container, so run this command after logging toui
containerapt update && apt -y install dnsutils telnet