Yolean/kubernetes-kafka

Consumer can not get messages

Closed this issue · 5 comments

Although it says that consumer connects to broker, my consumer app(which is a spring based web app) can not get messages from kafka inside same k8s cluster.
Then i tried to run my docker image at localhost but it does not work either.
So then again i tried to run my consumer app at localhost without docker it works like expected.

Any advice ?

I guess the usual stuff, enable debug logging and see what it says. Possible cause from this repo is min.insync.replicas=2 in broker config. Messages will only be sent to consumers once producers got two acks, which means you need two working brokers.

i am a total kafka newbie. So what i do not understand is why it is working when i run java -jar app.jar but it is not working when i run docker run pasali/app. Is this issue can be docker related?
Both of the cases consumer attached to partition, but the one runs inside docker can not get any messages

Broker addresses matter, much more than is common with REST. Typically kafka and your client must be able to resolve the same addresses, or have a look at the "advertised.listeners". With clients logging at info level they will appear hanged if kafka bootstrapping returns an address they fail to connect to. Could be the case since you see different results in different runtime environments.

@solsson thanks for your help. My issue was my docker image inherits from openjdk:9-jdk-slim
and apparently some packages that kafka uses are missing. So i changed it to openjdk:9-jdk and it works fine now.

BTW do you have any idea what cause this ? @solsson