[QUESTION]Save files in subfolder / Manage retention variables(time, size)
Closed this issue · 1 comments
perebaj commented
It's possible to change the time that dump is made into the GCS bucket?
I see that every 3 minutes the files are saved and the only variable that I can change is file.max.records
But I thought in control of the file size(MB size) and dump at minimum time.
Exists a way to save files in subfolder into bucket? Instead of use an entire?
LarsKlingen commented
To save files in subfolders you can modify the file properties. E.g. this would save it in a folder structure like bucket:/folder/topic/year/month/day/
"file.name.prefix": "folder/",
"file.name.template": "{{topic}}/{{timestamp:unit=yyyy}}/{{timestamp:unit=MM}}/{{timestamp:unit=dd}}/{{partition}}-{{start_offset}}.json",