how does delete_consumer work?
Closed this issue · 5 comments
HI,
I have tried to delete a consumer group from kafka with kafka-consumer-manager:
./kafka-consumer-manager --cluster-type pro delete_group console-consumer-68578
Cluster name: global-pro, consumer group: console-consumer-68578
Offsets of all topics and partitions listed below shall be modified:
Topic: scmspain.ms-ij-notifications.notification, Partitions: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29]
Is this what you really intend? (y/n) y
I can see my console-consumer-68578 in kafka-manager yet.
Musn't the script clean all references?
How does it work really?
Regards
This script only manipulates consumer offsets. Its behavior depends on the offset storage used:
--storage zookeeper: the offsets will be removed.--storage kafka(default): the script will commit a 0 offset, since Kafka doesn't provide an API to delete a committed offset explicitly. The offset will actually be removed when it will expire, which depends on the broker configuration.
Other group related information, such as active members, cannot be controlled via this script.
thanks, @fede1024, do you know how I can delete a consumer that I don't use?
I have read that with kafka will apply offsets.retention.minutes and offset will be 0, then will clean consumers from his topic_consumer.
the monitoring tool reads the offset-log to collect the information it shows. Thus, I would assume, that deleting the offsets is the same a deleting the consumer group.
do you know if is this right?
Consumer group membership will be updated as consumers join and leave the group, so if you don't have any consumer running there should be no group information. For the offsets, yes the configuration parameter you mention regulates their expiration, and waiting for the expiration is the only way to completely remove the offset.
Thanks @fede1024
What is the reasoning behind setting 0 offset with storage kafka?