Yelp/kafka-utils

Genetic Rebalance should raise RebalanceError when there are inactive brokers

Opened this issue · 0 comments

Right now Genetic Rebalance fails with ValueError when there are any inactive brokers.

Traceback (most recent call last):
  File "/usr/bin/kafka-cluster-manager", line 21, in <module>
    main.run()
  File "/opt/venvs/yelp-kafka-tool/local/lib/python2.7/site-packages/kafka_utils/kafka_cluster_manager/main.py", line 240, in run
    args,
  File "/opt/venvs/yelp-kafka-tool/local/lib/python2.7/site-packages/kafka_utils/kafka_cluster_manager/cmds/command.py", line 99, in run
    self.run_command(ct, cluster_balancer(ct, args))
  File "/opt/venvs/yelp-kafka-tool/local/lib/python2.7/site-packages/kafka_utils/kafka_cluster_manager/cmds/rebalance.py", line 184, in run_command
    base_score = cluster_balancer.score()
  File "/opt/venvs/yelp-kafka-tool/local/lib/python2.7/site-packages/kafka_utils/kafka_cluster_manager/cluster_info/genetic_balancer.py", line 458, in score
    return self._score(_State(self.cluster_topology), score_movement=False)
  File "/opt/venvs/yelp-kafka-tool/local/lib/python2.7/site-packages/kafka_utils/kafka_cluster_manager/cluster_info/genetic_balancer.py", line 731, in __init__
    self.rgs.index(broker.replication_group) for broker in self.brokers
  File "/opt/venvs/yelp-kafka-tool/local/lib/python2.7/site-packages/kafka_utils/kafka_cluster_manager/cluster_info/genetic_balancer.py", line 731, in <genexpr>
    self.rgs.index(broker.replication_group) for broker in self.brokers
ValueError: tuple.index(x): x not in tuple

We should check for inactive brokers and raise RebalanceError when they exist, as we do in PartitionCountBalancer.