snowdrop/vertx-spring-boot

Kafka starter - multiple clients problem

mocenas opened this issue · 5 comments

mocenas commented

Connecting 2 pods with Webflux Kafka client to Kafka broker seems to cause problems. Testing with 1 pod, all is OK. Testing with 2 pods (different app, different Kafka topic) causes one of the pods to have some error in the log (full log springboot-webflux-kafka-1-54g55.log):

2020-04-22 08:11:16.758 ERROR 1 --- [nsumer-thread-0] o.a.k.c.c.internals.ConsumerCoordinator : [Consumer clientId=consumer-1, groupId=log] Offset commit failed on partition rhoarqe-topic3-0 at offset 0: The coordinator is not aware of this member.
2020-04-22 08:11:16.759  WARN 1 --- [nsumer-thread-0] o.a.k.c.c.internals.ConsumerCoordinator  : [Consumer clientId=consumer-1, groupId=log] Asynchronous auto-commit of offsets {rhoarqe-topic3-0=OffsetAndMetadata{offset=0, metadata=''}} failed: Commit cannot be completed since the group has already rebalanced and assigned the partitions to another member. This means that the time between subsequent calls to poll() was longer than the configured max.poll.interval.ms, which typically implies that the poll loop is spending too much time message processing. You can address this either by increasing the session timeout or by reducing the maximum size of batches returned in poll() with max.poll.records.

Also this pod does occasionally miss some Kafka messages. In my test I've sent messages 1,2, it received only message 2.

gytis commented

@mocenas do you have a setup that I could use to reproduce this?

gytis commented

In your first application you manually assign a partition to a consumer. Therefore that consumer should not be in a consumer group. Remove vertx.kafka.consumer.group.id=log from the application.properties in that application.

Consumers groups are used to automatically organise which consumer will receive messages from which topic partition.

If you want to keep both your consumers in a same group, use subscribe instead of assign.

gytis commented

I'll add a note about this in a readme.

mocenas commented

@gytis Thanks a lot, I overlooked the consumer groups configuration entirely. I thought that if these applications have different topics, they should not interfere. Well I will study about the consumer groups.

Changing from assign to subscribe a giving a separate consumer group seems to solve the problem. Thanks.

  • 🎉1