Use KafkaAdminClient for quota enforcement + upgrade Kafka libs to 2.6
Opened this issue · 0 comments
danielfritsch commented
With the current Kafka version (2.4.1), quota enforcement was implemented through the use of a Zookeeper admin client, as using the KafkaAdminClient only support quota configuration with Kafka >= 2.6, client and server-side.
With the introduction of quota enforcement functionality in this project, we had to add in the Kafka server library (which contains the ZK admin client code), which in turn required complicating the dependency environment with various Scala libraries, and scala bazel_rules.
When we are ready to upgrade Kafka to >= 2.6, it would make sense to bring remove these Scala dependencies and go back to a light-weight dependencies.yaml with just Java libraries. This entails
- removing Kafka server library
- upgrading kafka-client library
- remove unneeded dependencies that were added in the below PRs
- use KafkaAdminClient to for quota configuration