Aiven-Open/gcs-connector-for-apache-kafka

Connector not detected in confluent Docker image

Closed this issue · 1 comments

drinkbird commented 
Kafka Connect Version: 5.5.2
Docker Image: confluentinc/cp-kafka-connect:5.5.2
Aiven Connector Version: 0.6.1

Although the connector plugin is loaded, according to the logs, it doesn't come up in the list of available plugins at [api]/connector-plugins.

To verify that the configuration works, I've added a second plugin (debezium) which is loaded correctly and it shows up in the list of available connectors.

Logs excerpt:

[2020-12-22 13:23:05,015] INFO Scanning for plugin classes. This might take a moment ... (org.apache.kafka.connect.cli.ConnectDistributed)
[2020-12-22 13:23:05,042] INFO Loading plugin from: /connectors/aiven-kafka-connect-gcs-0.6.1 (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-12-22 13:23:06,128] INFO Registered loader: PluginClassLoader{pluginLocation=file:/connectors/aiven-kafka-connect-gcs-0.6.1/} (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-12-22 13:23:06,128] INFO Added plugin 'org.apache.kafka.connect.connector.policy.AllConnectorClientConfigOverridePolicy' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-12-22 13:23:06,128] INFO Added plugin 'org.apache.kafka.connect.connector.policy.PrincipalConnectorClientConfigOverridePolicy' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-12-22 13:23:06,128] INFO Added plugin 'org.apache.kafka.connect.connector.policy.NoneConnectorClientConfigOverridePolicy' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-12-22 13:23:06,129] INFO Loading plugin from: /connectors/debezium-connector-postgres (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-12-22 13:23:06,371] INFO Registered loader: PluginClassLoader{pluginLocation=file:/connectors/debezium-connector-postgres/} (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-12-22 13:23:06,371] INFO Added plugin 'io.debezium.connector.postgresql.PostgresConnector' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-12-22 13:23:06,371] INFO Added plugin 'io.debezium.converters.ByteBufferConverter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-12-22 13:23:06,371] INFO Added plugin 'io.debezium.converters.CloudEventsConverter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-12-22 13:23:06,371] INFO Added plugin 'io.debezium.transforms.ExtractNewRecordState' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-12-22 13:23:06,371] INFO Added plugin 'io.debezium.transforms.outbox.EventRouter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-12-22 13:23:06,371] INFO Added plugin 'io.debezium.transforms.ByLogicalTableRouter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-12-22 13:23:10,969] INFO Registered loader: sun.misc.Launcher$AppClassLoader@18b4aac2 (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-12-22 13:23:10,970] INFO Added plugin 'org.apache.kafka.connect.tools.MockConnector' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)

I suspect the issue lies in the Java version, as the Confluent docker image uses:

openjdk version "1.8.0_222"
OpenJDK Runtime Environment (Zulu 8.40.0.25-CA-linux64) (build 1.8.0_222-b10)
OpenJDK 64-Bit Server VM (Zulu 8.40.0.25-CA-linux64) (build 25.222-b10, mixed mode)

The connector works in standalone mode though, within the same image.
EDIT: The connector we tried in standalone mode was 0.4.0, which also shows up in the connectors list. The Java 11 version seems to be the issue.

ivanyu commented

Hi,

Sorry for the late reply. Indeed, the connector targets Java 11.
I'm closing the issue, please feel free to re-open if we can provide more information.

  • 👍1