Support for newer kafka versions

[Hiteshitaneja]

GCS sink connector only works with kafka 1.1.1 which was released in july 2018. Is there a workaround to use it with newer versions of kafka atleast vesrion 2.7+?

[ivanyu]

Hi!

1.1.1 is the minimal Kafka version that supports the connector. However, it works perfectly on any modern Kafka Connect.

[Hiteshitaneja]

I tried out the connector on local machine for kafka version (1.1.1, 2.4.0, 2.6.2, 2.7.1, 2.8.0, 3.10) but It is working with just version 1.1.1. I downloaded the tar file provided in project and untar it into lib folder and plugins folder, then added the plugins directory into connect-standalone.properties and create the gcs-sink-connector.properties file referencing the documentation. But It is failing due to "java.util.concurrent.ExecutionException: java.lang.NoClassDefFoundError:", for different classes for each version. Attaching the connect-standalone.properties, gcs-sink-connector.properties files and error log for version 3.1.0.

Please let me know if I am doing anything wrong to setup the gcs sink connector.
connect-standalone.properties.txt
gcs-sink-connector.properties.txt
connect.log

[Hiteshitaneja]

Got the solution,needed to build a uber jar by checking out the project and making minor changes in build.gradle file.

For future reference, just the following snippet in jar section of build.gradle

from {
configurations.runtimeClasspath.collect { it.isDirectory() ? it : zipTree(it) }
}
duplicatesStrategy = DuplicatesStrategy.INCLUDE