Quotas not working out the box
sionsmith opened this issue · 1 comments
sionsmith commented
Describe the bug
Reading the docs I should be able set quotas using the YAML examples provided in the documentation:
platform:
kafka:
quotas:
- principal: "User:App0"
producer_byte_rate: 1024
consumer_byte_rate: 1024
request_percentage: 50.0
- principal: "User:App1"
producer_byte_rate: 2048
consumer_byte_rate: 2048
- principal: "User:App2"
request_percentage: 80.0
However quotas is not supported which throws the following error:
➜ julie git:(jenkins) ✗ java -jar julie-ops.jar --brokers localhost:29092 --clientConfig sasl_plain.properties --topology sasl-descriptor-topics-quotas.yaml --dryRun
log4j:WARN No appenders could be found for logger (org.apache.kafka.clients.admin.AdminClientConfig).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
com.purbon.kafka.topology.exceptions.TopologyParsingException: Failed to deserialize topology from sasl-descriptor-topics-quotas.yaml
at com.purbon.kafka.topology.serdes.TopologySerdes.deserialise(TopologySerdes.java:42)
at com.purbon.kafka.topology.TopologyObjectBuilder.parseListOfTopologies(TopologyObjectBuilder.java:98)
at com.purbon.kafka.topology.TopologyObjectBuilder.build(TopologyObjectBuilder.java:36)
at com.purbon.kafka.topology.TopologyObjectBuilder.build(TopologyObjectBuilder.java:30)
at com.purbon.kafka.topology.JulieOps.build(JulieOps.java:127)
at com.purbon.kafka.topology.JulieOps.build(JulieOps.java:87)
at com.purbon.kafka.topology.CommandLineInterface.processTopology(CommandLineInterface.java:211)
at com.purbon.kafka.topology.CommandLineInterface.run(CommandLineInterface.java:161)
at com.purbon.kafka.topology.CommandLineInterface.main(CommandLineInterface.java:147)
Caused by: com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException: Unrecognized field "quotas" (class com.purbon.kafka.topology.model.users.platform.Kafka), not marked as ignorable (2 known properties: "rbac", "instances"])
at [Source: UNKNOWN; line: -1, column: -1] (through reference chain: com.purbon.kafka.topology.model.users.platform.Kafka["quotas"])
at com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException.from(UnrecognizedPropertyException.java:61)
at com.fasterxml.jackson.databind.DeserializationContext.handleUnknownProperty(DeserializationContext.java:987)
at com.fasterxml.jackson.databind.deser.std.StdDeserializer.handleUnknownProperty(StdDeserializer.java:1974)
at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.handleUnknownProperty(BeanDeserializerBase.java:1701)
at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.handleUnknownVanilla(BeanDeserializerBase.java:1679)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:330)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:187)
at com.fasterxml.jackson.databind.deser.DefaultDeserializationContext.readRootValue(DefaultDeserializationContext.java:322)
at com.fasterxml.jackson.databind.ObjectMapper._readValue(ObjectMapper.java:4569)
at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2798)
at com.fasterxml.jackson.databind.ObjectMapper.treeToValue(ObjectMapper.java:3261)
at com.purbon.kafka.topology.serdes.TopologyCustomDeserializer.parse(TopologyCustomDeserializer.java:177)
at com.purbon.kafka.topology.serdes.TopologyCustomDeserializer.deserialize(TopologyCustomDeserializer.java:131)
at com.purbon.kafka.topology.serdes.TopologyCustomDeserializer.deserialize(TopologyCustomDeserializer.java:47)
at com.fasterxml.jackson.databind.deser.DefaultDeserializationContext.readRootValue(DefaultDeserializationContext.java:322)
at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4593)
at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3413)
at com.purbon.kafka.topology.serdes.TopologySerdes.deserialise(TopologySerdes.java:39)
... 8 more
To Reproduce
java -jar julie-ops.jar --brokers localhost:29092 --clientConfig sasl_plain.properties --topology /julie/src/test/resource/descriptor-with-quotas.yaml
Expected behavior
A quota would be added to the user
Runtime (please complete the following information):
- OS: mac
- JVM version: Java 17
- JulieOps Version 4.3.0
Additional context
Add any other context about the problem here.
sionsmith commented
I have just noticed that Quotas were not added until 4.4.0 my bad!