scylladb/kafka-connect-scylladb

Is it possible to use dynamic keyspace?

tunix opened this issue · 1 comments

tunix commented

Hi,

I've a multi-tenant application. I want to read messages from a single Kafka topic and split messages across different scylladb keyspaces based on a field inside the Kafka message.

Does the connector support this? If not, would you consider supporting it?

Currently, it is not supported.

As a workaround, assuming the number of destination keyspaces is small and fixed, you could launch N instances of Sink Connector (each one writing to a specific keyspace) and use a Filter (Confluent version) SMT transformation on each connector instance to filter the messages to a correct keyspace.

We will consider adding this feature in the future.