morganstanley/modern-cpp-kafka

[librdkafka issue] rd_kafka_consumer_close() would keep retrying the "coordinator query" if there's no response from brokers -- thus the consumer would stuck there forever

kenneth-jia opened this issue · 0 comments

  • A placeholder
    librdkafka issue: confluentinc/librdkafka#3740

  • Description
    We found the PR confluentinc/librdkafka#3416 has changed the retry behavior for "offset commit".
    If a consumer try to commit offsets with rd_kafka_commit_queue(...) before closing (with rd_kafka_consumer_close(...)), but there's no response from brokers, the consumer would be keep retrying the "coordinator query" forever, thus the consumer would stuck there (and user's application could never quit)!

  • How to reproduce

    • Robustness test: TEST(KafkaConsumer, DISABLED_AlwaysFinishClosing_ManuallyPollEvents)
    • Robustness test: TEST(KafkaConsumer, DISABLED_CommitOffsetWhileBrokersStop)