Kafka Channel could not connect to off-cluster Kafka
Yiyiyimu opened this issue · 9 comments
Describe the bug
In Kafka Channel Deployment step, it says we could use off-cluster Kafka. But it seems I could not do it. I tried to change bootstrapServers to localhost:9092, 127.0.0.1:9092, 0.0.0.0:9092, host.minikube.internal:9092(Reference), host.docker.internal:9092(Reference) but none of them seems work. And the error comes from failure of creating client like:
InvalidConfiguration: Unable to build Kafka admin client for channel my-kafka-channel: kafka: client has run out of available brokers to talk to (Is your cluster reachable?)
Expected behavior
Kafka Channel could connect to local kafka cluster
To Reproduce
- install and test kafka following kafka quick start: https://kafka.apache.org/quickstart
- install eventing components following https://knative.dev/docs/install/any-kubernetes-cluster/
- create kafkachannel and
kubectl describe kafkachannel
Knative release version
0.17
Additional context
OS: WSL2
k8s: minikube, driver = docker
Hi @Yiyiyimu, using minikube you cannot access to your local resources on your machine. I think there are some threads here and there that explains if this is possible and how kubernetes/minikube#2735, but that really depends on your kube env and not on knative
@slinkydeveloper Thank you for your prompt reply! Actually this issue thread just updated this June that we could access local service as shown here: kubernetes/minikube#2735 (comment). But sadly it's not compatible with docker based minikube...
But anyway, do you have any suggestions on which kind of k8s to use in need of connecting local kafka?
TBH i have no clue, i always use strimzi and start kafka directly in kube
@slinkydeveloper Still thanks! The situation is I'm developing RocketMQ channel (as I asked for help before in #1370 ). The RocketMQ Operator is not so well crafted so I'm trying to connect to local cluster, and found it seems not working for connecting local kafka also.
The information suggests that we could connect to off-cluster kafka was added at the beginning of kafka channel (#499). @matzew do you have any suggestions?
Yes you can connect to whatever kafka you want, but it must be reachable by the internal kubernetes network, so it's not an issue specific to kafka channel or knative itself, but it's something that depends on your kubernetes distro (in this case, minikube). Same for rocketmq or any other external service
Got it. Thank you.
and still we can't connect to a service on the hosting host... something that really should be trivial to configure.
I'm battling in that on my MBP (running minikube in docker) I don't have enough resources to run Csssandra, postgresql and others inside minikube, but I can run them out on the host, so need to connect to them from within the minikube environment, it's another 8 months after the above updates and on the other threads and still nothing :(
G