not able to use S3 source connector

[userakhila]

Hi,
I am seeing below issue when running connector. and here is my configuration, I am not using Hadoop configuration in my config but still its failing with Hadoop error, can someone please help me resolve this issue?

config -

   "name": 'test_s3_source',"config": {
        "connector.class" : "com.github.mmolimar.kafka.connect.fs.FsSourceConnector",
        "tasks.max":  1,
        "topic":  "test.s3.source",
        "fs.uris" : "s3://s3-bucket",
        "policy.class" : "com.github.mmolimar.kafka.connect.fs.policy.SimplePolicy",
        "policy.regexp" : ".* \.json$",
        "file_reader.class" : "com.github.mmolimar.kafka.connect.fs.file.reader.JsonFileReader"
       }

Error -

Task threw an uncaught and unrecoverable exception (org.apache.kafka.connect.runtime.WorkerTask:179)
org.apache.kafka.connect.errors.ConnectException: A problem has occurred reading configuration: java.lang.NoClassDefFoundError: org/apache/hadoop/conf/Configuration
	at com.github.mmolimar.kafka.connect.fs.FsSourceTask.start(FsSourceTask.java:68)
	at org.apache.kafka.connect.runtime.WorkerSourceTask.execute(WorkerSourceTask.java:199)
	at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:177)
	at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:227)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: org.apache.kafka.connect.errors.ConnectException: java.lang.NoClassDefFoundError: org/apache/hadoop/conf/Configuration
	at com.github.mmolimar.kafka.connect.fs.util.ReflectionUtils.make(ReflectionUtils.java:36)
	at com.github.mmolimar.kafka.connect.fs.util.ReflectionUtils.makePolicy(ReflectionUtils.java:24)
	at com.github.mmolimar.kafka.connect.fs.FsSourceTask.start(FsSourceTask.java:61)
	... 8 more
Caused by: java.lang.NoClassDefFoundError: org/apache/hadoop/conf/Configuration

[mmolimar]

Hi.
That class belongs to hadoop-common lib and that error is due to you don't have the connector dependencies included in your classpath.
When you package the connector, Maven creates a folder with the all required dependencies and all of them should be included in the Kafka Connect classpath.