Kafka User deletion failed
shubhamcoc opened this issue · 2 comments
shubhamcoc commented
Description
If the Kafka User is created with ACLs, the deletion of the Kafka user is failing.
Banzaicloud koperator logs says "failed to finalize kafkauser" with error EOF.
{"level":"info","ts":"2023-05-24T11:20:53.218Z","msg":"Reconciling KafkaUser","controller":"KafkaUser","controllerGroup":"kafka.banzaicloud.io","controllerKind":"KafkaUser","KafkaUser":{"name":"example-kafkauser-with-pki","namespace":"default"},"namespace":"default","name":"example-kafkauser-with-pki","reconcileID":"d1c7e23c-cf67-47f3-8675-c8c6b6579a68"}
{"level":"info","ts":"2023-05-24T11:20:53.223Z","msg":"Kafka user is marked for deletion, revoking certificates","controller":"KafkaUser","controllerGroup":"kafka.banzaicloud.io","controllerKind":"KafkaUser","KafkaUser":{"name":"example-kafkauser-with-pki","namespace":"default"},"namespace":"default","name":"example-kafkauser-with-pki","reconcileID":"d1c7e23c-cf67-47f3-8675-c8c6b6579a68"}
{"level":"info","ts":"2023-05-24T11:20:53.223Z","msg":"Deleting user ACLs from kafka","controller":"KafkaUser","controllerGroup":"kafka.banzaicloud.io","controllerKind":"KafkaUser","KafkaUser":{"name":"example-kafkauser-with-pki","namespace":"default"},"namespace":"default","name":"example-kafkauser-with-pki","reconcileID":"d1c7e23c-cf67-47f3-8675-c8c6b6579a68"}
{"level":"info","ts":"2023-05-24T11:20:53.223Z","msg":"Creating kafka client connection","controller":"KafkaUser","controllerGroup":"kafka.banzaicloud.io","controllerKind":"KafkaUser","KafkaUser":{"name":"example-kafkauser-with-pki","namespace":"default"},"namespace":"default","name":"example-kafkauser-with-pki","reconcileID":"d1c7e23c-cf67-47f3-8675-c8c6b6579a68"}
{"level":"info","ts":"2023-05-24T11:20:54.180Z","msg":"broker deleteusersacls called","controller":"KafkaUser","controllerGroup":"kafka.banzaicloud.io","controllerKind":"KafkaUser","KafkaUser":{"name":"example-kafkauser-with-pki","namespace":"default"},"namespace":"default","name":"example-kafkauser-with-pki","reconcileID":"d1c7e23c-cf67-47f3-8675-c8c6b6579a68"}
{"level":"info","ts":"2023-05-24T11:20:54.183Z","logger":"kafka_util","msg":"Kafka client closed cleanly"}
{"level":"info","ts":"2023-05-24T11:20:54.183Z","msg":"failed to finalize kafkauser","controller":"KafkaUser","controllerGroup":"kafka.banzaicloud.io","controllerKind":"KafkaUser","KafkaUser":{"name":"example-kafkauser-with-pki","namespace":"default"},"namespace":"default","name":"example-kafkauser-with-pki","reconcileID":"d1c7e23c-cf67-47f3-8675-c8c6b6579a68"}
{"level":"error","ts":"2023-05-24T11:20:54.183Z","msg":"Reconciler error","controller":"KafkaUser","controllerGroup":"kafka.banzaicloud.io","controllerKind":"KafkaUser","KafkaUser":{"name":"example-kafkauser-with-pki","namespace":"default"},"namespace":"default","name":"example-kafkauser-with-pki","reconcileID":"d1c7e23c-cf67-47f3-8675-c8c6b6579a68","error":"EOF","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.13.0/pkg/internal/controller/controller.go:326\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.13.0/pkg/internal/controller/controller.go:273\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.13.0/pkg/internal/controller/controller.go:234"}
Kafka brokers logs
[2023-05-25 05:36:37,003] INFO [Producer clientId=CruiseControlMetricsReporter] Node 0 disconnected. (org.apache.kafka.clients.NetworkClient) │
│ [2023-05-25 05:39:42,671] ERROR Closing socket for XXX because of error (kafka.network.Processor) │
│ org.apache.kafka.common.errors.InvalidRequestException: Error getting request for apiKey: DELETE_ACLS, apiVersion: 1, connectionId: XXX, listenerName: ListenerNam │
│ Caused by: java.lang.IllegalArgumentException: Filters contain UNKNOWN elements, filters: [DeleteAclsFilter(resourceTypeFilter=0, resourceNameFilter=null, patternTypeFilter=0, principalFilter='CN=exa │
│ at org.apache.kafka.common.requests.DeleteAclsRequest.normalizeAndValidate(DeleteAclsRequest.java:94) │
│ at org.apache.kafka.common.requests.DeleteAclsRequest.<init>(DeleteAclsRequest.java:67) │
│ at org.apache.kafka.common.requests.DeleteAclsRequest.parse(DeleteAclsRequest.java:120) │
│ at org.apache.kafka.common.requests.AbstractRequest.doParseRequest(AbstractRequest.java:233) │
│ at org.apache.kafka.common.requests.AbstractRequest.parseRequest(AbstractRequest.java:165) │
│ at org.apache.kafka.common.requests.RequestContext.parseRequest(RequestContext.java:95) │
│ at kafka.network.RequestChannel$Request.<init>(RequestChannel.scala:101) │
│ at kafka.network.Processor.$anonfun$processCompletedReceives$1(SocketServer.scala:1030) │
│ at java.base/java.util.LinkedHashMap$LinkedValues.forEach(Unknown Source) │
│ at kafka.network.Processor.processCompletedReceives(SocketServer.scala:1008) │
│ at kafka.network.Processor.run(SocketServer.scala:893) │
│ at java.base/java.lang.Thread.run(Unknown Source) │
│ [2023-05-25 05:39:42,672] ERROR Exception while processing request from XXX (kafka.network.Processor) │
│ org.apache.kafka.common.errors.InvalidRequestException: Error getting request for apiKey: DELETE_ACLS, apiVersion: 1, connectionId: XXX, listenerName: ListenerNam │
│ Caused by: java.lang.IllegalArgumentException: Filters contain UNKNOWN elements, filters: [DeleteAclsFilter(resourceTypeFilter=0, resourceNameFilter=null, patternTypeFilter=0, principalFilter='CN=exa │
│ at org.apache.kafka.common.requests.DeleteAclsRequest.normalizeAndValidate(DeleteAclsRequest.java:94) │
│ at org.apache.kafka.common.requests.DeleteAclsRequest.<init>(DeleteAclsRequest.java:67) │
│ at org.apache.kafka.common.requests.DeleteAclsRequest.parse(DeleteAclsRequest.java:120) │
│ at org.apache.kafka.common.requests.AbstractRequest.doParseRequest(AbstractRequest.java:233) │
│ at org.apache.kafka.common.requests.AbstractRequest.parseRequest(AbstractRequest.java:165) │
│ at org.apache.kafka.common.requests.RequestContext.parseRequest(RequestContext.java:95) │
│ at kafka.network.RequestChannel$Request.<init>(RequestChannel.scala:101) │
│ at kafka.network.Processor.$anonfun$processCompletedReceives$1(SocketServer.scala:1030) │
│ at java.base/java.util.LinkedHashMap$LinkedValues.forEach(Unknown Source) │
│ at kafka.network.Processor.processCompletedReceives(SocketServer.scala:1008) │
│ at kafka.network.Processor.run(SocketServer.scala:893) │
│ at java.base/java.lang.Thread.run(Unknown Source)
This issue was also raised in #728, but it is closed.
Expected Behavior
Kafka User should be deleted.
Actual Behavior
Kafka User deletion failed with error EOF
Affected Version
Koperator: 0.24. 1
Kafka: ghcr.io/banzaicloud/kafka:2.13-3.1.0
Steps to Reproduce
- Create kafka cluster with ACL enabled
- Create kafka topic
- Create Kafka user with ACLs
- Delete Kafka user
Checklist
- I have read the contributing guidelines
panyuenlau commented
Thanks for reporting this @shubhamcoc, just quickly checked and confirmed this is a bug on the Koperator.
Will review the PR early next week