apache/camel-kafka-connector-examples

Example how to export from Kafka to S3 and import back from S3 to Kafka both key and value

gohanbg opened this issue · 2 comments

gohanbg commented

Hello,

I think you are doing a great job with all the kafka connectors that you are building!

Some time ago I played with the org.apache.camel.kafkaconnector.awss3.CamelAwss3SinkConnector connector and successfully managed to export records from my topic to S3.

I now want to be able to import the records from S3 back to kafka. That is fairly straight forward, but there is one tricky part - I want to keep the keys.

For example if I had a record with

key="key_UUID"
value="value_JSON"

I want to export both the key and the value, and then be able to create the same record with the same key and same value. Is this possible, and if so can you give me an example configuration for the S3 Sink/Source.

Thanks
Mihail

oscerd commented

You can have a look at this SMT

https://github.com/apache/camel-kafka-connector/pull/1204/files

It's still not merged in the code base.

gohanbg commented

Thanks @oscerd.