Aiven-Open/gcs-connector-for-apache-kafka

java.lang.ClassNotFoundException: com.google.auth.Credentials

Closed this issue · 5 comments

wengkham commented

Hi, was trying out this connector but bump into error below, appreciate if you can shed some lights on this error java.lang.NoClassDefFoundError: com/google/auth/Credentials

logs as followed:

debezium_1         | 2020-04-17 06:27:24,699 ERROR  ||  Failed to start task auction-gcs-sink-0   [org.apache.kafka.connect.runtime.Worker]                                        
debezium_1         | java.lang.NoClassDefFoundError: com/google/auth/Credentials                                                                                                   
debezium_1         |    at java.base/java.lang.Class.forName0(Native Method)                                                                                                       
debezium_1         |    at java.base/java.lang.Class.forName(Class.java:398)                                                                                                       
debezium_1         |    at org.apache.kafka.common.config.ConfigDef.parseType(ConfigDef.java:719)                                                                                  
debezium_1         |    at org.apache.kafka.common.config.ConfigDef.parseValue(ConfigDef.java:474)                                                                                 
debezium_1         |    at org.apache.kafka.common.config.ConfigDef.parse(ConfigDef.java:467)                                                                                      
debezium_1         |    at org.apache.kafka.common.config.AbstractConfig.<init>(AbstractConfig.java:108)                                                                           
debezium_1         |    at org.apache.kafka.common.config.AbstractConfig.<init>(AbstractConfig.java:142)                                                                           
debezium_1         |    at org.apache.kafka.connect.runtime.TaskConfig.<init>(TaskConfig.java:51)                                                                                  
debezium_1         |    at org.apache.kafka.connect.runtime.Worker.startTask(Worker.java:431)                                                                                      
debezium_1         |    at org.apache.kafka.connect.runtime.distributed.DistributedHerder.startTask(DistributedHerder.java:1140)                                                   
debezium_1         |    at org.apache.kafka.connect.runtime.distributed.DistributedHerder.access$1700(DistributedHerder.java:125)                                                  
debezium_1         |    at org.apache.kafka.connect.runtime.distributed.DistributedHerder$13.call(DistributedHerder.java:1155)                                                     
debezium_1         |    at org.apache.kafka.connect.runtime.distributed.DistributedHerder$13.call(DistributedHerder.java:1151)                                                     
debezium_1         |    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)                                                                                      
debezium_1         |    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)                                                               
debezium_1         |    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)                                                               
debezium_1         |    at java.base/java.lang.Thread.run(Thread.java:834)                                                                                                         
debezium_1         | Caused by: java.lang.ClassNotFoundException: com.google.auth.Credentials                                                                                      
debezium_1         |    at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:471)                                                                                    
debezium_1         |    at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:588)                                                                                         
debezium_1         |    at org.apache.kafka.connect.runtime.isolation.PluginClassLoader.loadClass(PluginClassLoader.java:104)                                                      
debezium_1         |    at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521)                                                                                         
debezium_1         |    ... 17 more

my config as followed:

{
"name": "auction-gcs-sink",
  "config": {
    "connector.class": "io.aiven.kafka.connect.gcs.GcsSinkConnector",
    "tasks.max": "1",
    "key.converter": "org.apache.kafka.connect.converters.ByteArrayConverter",
    "value.converter": "org.apache.kafka.connect.converters.ByteArrayConverter",
    "topics": "some_topic",
    "gcs.credentials.path": "/path/to/google_credentials.json",
    "gcs.bucket.name": "some-bucket",
    "file.name.prefix": "some-prefix/",
    "file.compression.type": "gzip",
    "format.output.fields": "key,value,offset"
  }
}
ivanyu commented

Hi,

Could you please specify which release you're trying to use?

wengkham commented

Hi, thanks for the reply
I am using the latest release v0.3.0

ivanyu commented

I noticed that there are no build attached to the release. (We'll fix this shortly!) This mean you must have built the connector by yourself. Could you please try running ./gradlew clean distTar and using the whole content of build/distributions/aiven-kafka-connect-gcs-0.3.0.tar in your Kafka Connect? It should contain all the necessary classes.

wengkham commented

Yup, I have built it with Gradle. Sorry for asking the obvious as I'm not a Java developer.

Thanks a lot for your replying, after copying all .jars in the .tar into Kafka Connect it seems the dependencies are resolved. Unfortunately, I am having this problem:

Failed to create GCS credentials: Failed to read GCS credentials from google_credentials.json

I have tried relative and full path to the credential file but having the same error. I am running Kafka and Kafka Connect locally. The conf as followed:

"gcs.credentials.path": "/path/to/google_credentials.json",

This should be something obvious and dumb and I missed it

wengkham commented

ok found the issue, the credentials files need to be inside the container, thanks for your help. Have a nice day