Published version in maven doesn't work with Confluent 3.3
Opened this issue · 5 comments
The kafka-connect-common version 0.8.2.7 in maven is a shadowJar and includes dependencies. It contains the com.google.common.* classes. These conflict with the Guava version used by Confluent 3.3 and cause the following error:
confluent-connect: Exception in thread "main" java.lang.NoSuchMethodError: com.google.common.collect.Sets$SetView.iterator()Lcom/google/common/collect/UnmodifiableIterator;
confluent-connect: at org.reflections.Reflections.expandSuperTypes(Reflections.java:380)
confluent-connect: at org.reflections.Reflections.<init>(Reflections.java:126)
confluent-connect: at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.scanPluginPath(DelegatingClassLoader.java:221)
confluent-connect: at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.scanUrlsAndAddPlugins(DelegatingClassLoader.java:198)
confluent-connect: at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.initLoaders(DelegatingClassLoader.java:159)
confluent-connect: at org.apache.kafka.connect.runtime.isolation.Plugins.<init>(Plugins.java:47)
confluent-connect: at org.apache.kafka.connect.cli.ConnectDistributed.main(ConnectDistributed.java:63)
confluent-connect.service: main process exited, code=exited, status=1/FAILURE
Unit confluent-connect.service entered failed state.
confluent-connect.service failed.
@joewood are you using the classloader plugin.path ?
No I'm deploying streamreactor's JARs from maven central to a directory now that streamreactor is in maven. Then just using CLASSPATH before kicking off the distributed connector process. Is there a better way?
Kafka 0.11 has classloader isolation. You can set the plugin. path in the worker properties to something like plugins/libs and drop the jars in there.
Yes, thanks - just testing with that: https://cwiki.apache.org/confluence/display/KAFKA/KIP-146+-+Classloading+Isolation+in+Connect
That fixed it, thanks! Out of interest - what's the driver for publishing a fat/shadowJar to Maven vs. just the compiled artifact?