Aiven-Open/gcs-connector-for-apache-kafka

Since this 26th, december "YYYY" template points to 2022 oO

sylvek opened this issue · 6 comments

I don't know how that is possible but, i currently have an issue on my kafka-exporter.
until 25th, december 2021, my template is perfectly parsed (YYYYMMDD => 20211225) but after this date, it returns 20221226 🤔 .

my current configuration:

curl -X POST http://localhost:8083/connectors \
-H "Content-Type: application/json" -d @- \
 << EOF
{
  "name": "herow-backbone-export",
  "config": {
    "name": "herow-backbone-export",
    "connector.class": "io.aiven.kafka.connect.gcs.GcsSinkConnector",
    "tasks.max": "1",
    "key.converter": "org.apache.kafka.connect.converters.ByteArrayConverter",
    "value.converter": "org.apache.kafka.connect.converters.ByteArrayConverter",
    "topics": "backbone-logs",
    "gcs.credentials.path": "/secrets/google-credentials.json",
    "gcs.bucket.name": "xxxx",
    "file.name.prefix": "backbone/",
    "file.compression.type": "gzip",
    "format.output.fields": "timestamp,key,value",
    "file.name.template": "{{topic}}-{{timestamp:unit=YYYY}}{{timestamp:unit=MM}}{{timestamp:unit=dd}}-{{timestamp:unit=HH}}-{{partition}}-{{start_offset:padding=false}}.gz",
    "file.name.timestamp.source": "WALLCLOCK",
    "file.name.timestamp.timezone": "Z"
  }
}
EOF

a "date" on my pod returns :

[appuser@backbone-export ~]$ date
Mon Dec 27 19:07:56 UTC 2021

I use the version 0.6.0 of your jar.

OC, I don't have this issue on my another platform.
I have exactly the same thing on my another platform :-o
Capture d’écran 2021-12-27 à 19 41 54

Hi @sylvek
Could you please tell me what version of the connector are you using?

Hi @ivanyu and thank you for your help.
My Docker image containing gcs-connector-for-apache-kafka comes from

FROM garthk/unzip AS unzipped
COPY aiven-kafka-connect-gcs-0.6.0.zip /tmp/aiven-kafka-connect-gcs.zip
RUN unzip -j /tmp/aiven-kafka-connect-gcs.zip -d /tmp/aiven-kafka-connect-gcs

FROM confluentinc/cp-kafka-connect-base:6.1.1
COPY --from=unzipped /tmp/aiven-kafka-connect-gcs /usr/share/java/aiven-kafka-connect-gcs

I started to read the code but not found where date parsing is set.

So, 0.6.0. Unfortunately, it's a bug that was fixed in 0.6.1.

ok i look that, thanks!

Yes, it's working but to be clear.. my file is now set to be an "application/octet" 🤔
Capture d’écran 2021-12-28 à 11 03 03

Everything seems OK. Thank you and merry Christmas.