Can't manipulate offsets in kafka storage

[boniek83]

`kafka-consumer-manager -t production --cluster-name kafka-1 offset_get --storage kafka test
Cluster name: kafka-1, consumer group: test

Traceback (most recent call last):
File "/usr/bin/kafka-consumer-manager", line 6, in
run()
File "/usr/lib/python2.7/site-packages/kafka_utils/kafka_consumer_manager/main.py", line 90, in run
args.command(args, conf)
File "/usr/lib/python2.7/site-packages/kafka_utils/kafka_consumer_manager/commands/offset_get.py", line 92, in run
storage=args.storage,
File "/usr/lib/python2.7/site-packages/kafka_utils/kafka_consumer_manager/commands/offset_manager.py", line 73, in preprocess_args
fail_on_error=fail_on_error,
File "/usr/lib/python2.7/site-packages/kafka_utils/kafka_consumer_manager/commands/offset_manager.py", line 39, in get_topics_from_consumer_group_id
return cls.get_topics_for_group_from_kafka(cluster_config, groupid)
File "/usr/lib/python2.7/site-packages/kafka_utils/kafka_consumer_manager/commands/offset_manager.py", line 133, in get_topics_for_group_from_kafka
return kafka_group_reader.read_group(groupid)
File "/usr/lib/python2.7/site-packages/kafka_utils/kafka_consumer_manager/util.py", line 168, in read_group
return self.read_groups(partition).get(group_id, [])
File "/usr/lib/python2.7/site-packages/kafka_utils/kafka_consumer_manager/util.py", line 183, in read_groups
consumer_timeout_ms=3000,
File "/usr/lib/python2.7/site-packages/kafka/consumer/kafka.py", line 60, in init
self.set_topic_partitions(*topics)
File "/usr/lib/python2.7/site-packages/kafka/consumer/kafka.py", line 224, in set_topic_partitions
self._consume_topic_partition(topic, partition)
File "/usr/lib/python2.7/site-packages/kafka/consumer/kafka.py", line 588, in _consume_topic_partition
"in broker metadata" % (partition, topic))
kafka.common.UnknownTopicOrPartitionError: Partition 35 not found in Topic __consumer_offsets in broker metadata`

I only have 20 partitions in __consumer_offsets topic. How on earth it finds at least 35 of them? :)
Tested on 0.9.0.1 and 0.10.1.0 clusters.

[fede1024]

The offset_get command expects the __consumer_offsets topic to have 50 partitions (the default value). At the moment this parameter is not configurable.

Version 0.5.6 doesn't rely on this parameter so you could temporarily roll back to that version.

[boniek83]

0.5.6 works, thanks.