[Feature request] specific naming pattern for created files

Question

[Feature request] specific naming pattern for created files

Closed this issue 5 years ago · 5 comments

IvanIschenko commented 5 years ago

Hi guys, thanks for your work!
I have two questions:

I am currently using Kafka GCS sink connector, and I am wondering can I somehow configure my specific naming pattern for created files (for example to take some value from the event).
Also, I am interested can I configure a write process of only one record per file in Kafka GCS sink connector.

Answer 1 · 2019-05-22T09:13:44.000Z

@IvanIschenko for 2, I created a separate issue (#9) and start working on it.

Answer 2 · 2019-05-22T09:14:38.000Z

@IvanIschenko about 1, could you please give an example of the naming pattern you would like to have?

Answer 3 · 2019-05-22T20:13:32.000Z

@ivanyu Currently, we are using <prefix><topic>-<partition>-<start-offset>[.gz] pattern for files.
Can we use key values from the message in Kafka topic (so I can paste in theory any value there)?

Answer 4 · 2019-06-10T11:56:42.000Z

@IvanIschenko FYI: #19 (mainly, README.md)

Answer 5 · 2019-06-10T15:28:35.000Z

I will close this task because of #19 task.