streamdal/plumber

[Kafka] Fail to emit cloudevents on scram authed cluster

WDaan opened this issue · 1 comments

Describe the bug
Emitting events with --encode-type=cloudevent works on an unauthenticated kafka-cluster but not on an authenticated one.

To Reproduce
Steps to reproduce the behavior:

  1. Launch XYZ message broker
  2. Execute the following command: plumber write kafka --topics=uptime --input-file=/home/event.json --ce-id=$(cat /proc/sys/kernel/random/uuid) --encode-type=cloudevent

Expected behavior
Expected the message to be sent.

Screenshots
If applicable, add screenshots to help explain your problem.

Emitting a cloudevent on an unauthenticated cluster

image

Emitting a cloud event on an authenticated cluster

Screenshot 2024-05-04 at 09 10 18

Omitting the encode-type, does work but message is no longer in the cloudevent format.

Also, all events in a non cloud-event format, even if you set the --key-option end up on the same partition 🤔

Screenshot 2024-05-04 at 09 13 36

CLI (please complete the following information):

  • OS: alpine:latest
  • Version: v2.6.1-e4b9db4

Additional context
Add any other context about the problem here.

In the examples an SCRAMClientGeneratorFunc is set. Might be a fix based on the log message https://github.com/IBM/sarama/blob/0ab2bb77aeca321f41a0953a8c6f52472607a59e/examples/sasl_scram_client/main.go#L96-L98