Question: What is a difference between two metrics `kminion_kafka_consumer_group_topic_offset_sum` and `kminion_kafka_consumer_group_offset_commits_total`
s2504s opened this issue · 2 comments
Hi there!
Firstly I want to say a big Thanks to you for this cool Open Source product!
Intro:
- One kafka cluster
- One topic
- One consumer-group
- One kminion-exporter
Could you please explain to me why I get different values of these two metrics at the same time:
rate(kminion_kafka_consumer_group_topic_offset_sum) = 35
rate(kminion_kafka_consumer_group_offset_commits_total) = 9
The documentation says me that:
kminion_kafka_consumer_group_topic_offset_sum is The sum of all committed group offsets across all partitions in a topic
kminion_kafka_consumer_group_offset_commits_total is The number of offsets committed by a group
If I have just one topic and one group then these values should be Identical.
Or not?
A consumer group commits their offsets to Kafka / Redpanda on a regular basis. It is up to the Kafka client how often these commits should happen. Some applications may send one offset commit per second per partition, others may be more frequent and want to commit the latest consumed offset after every individual kafka record hat had been consumed (not recommended).
Thus it is expected that these numbers are different. The metrics is mostly meant to help you spot issues with consumer groups committing too often (e.g. >1k / sec)