DataDog/kafka-kit

[topicmappr] rebalance relocation plans should be merged with constraints

jamiealquiza opened this issue · 0 comments

In rebalance, replica members scheduled in the rebalance planner are not accounted for in the constraints check. This could mean that in a very large partition rebalance, it's possible that two replicas for a partition could schedule a hand off to the same destination broker.

For instance, topic x p0 (partition) may consist of replicas [1001,1002]. In one pass, 1001 may schedule p0 to destination 1003. In another pass, 1002 may also schedule p0 to 1003, resulting in the replica set [1003,1003]. While this will fail to apply, this is a bug.

This fix would be to supply relocation scheduler info into the placement constraints mechanism. Additionally, an output partition map sanity check would help.