Exception in thread "main" java.lang.NoSuchMethodError:

Question

Exception in thread "main" java.lang.NoSuchMethodError:

Closed this issue 6 years ago · 14 comments

Exception in thread "main" java.lang.NoSuchMethodError: com.google.common.base.Preconditions.checkArgument(ZLjava/lang/String;Ljava/lang/Object;Ljava/lang/Object;)V
at com.google.cloud.hadoop.io.bigquery.BigQueryStrings.parseTableReference(BigQueryStrings.java:68)
at com.samelamin.spark.bigquery.BigQueryDataFrame.saveAsBigQueryTable(BigQueryDataFrame.scala:40)

Can you please tell me from which library you are getting 'checkArgument'?

Answer 1 · 2019-03-18T11:58:12.000Z

I am facing the same problem. I was using the library without problems in google Dataproc environtment and from 22 of febrary is failing on the last version of dataproc.

Answer 2 · 2019-03-18T12:17:02.000Z

Sorry folks I am away on Holiday till Friday

I believe that error is due to a guava version mismatch, if you google it you will see where the problem is

Same issue appears here #72

I think if you shade your libraries and create a fat jar it should work as long as guava is over version 14, i use 18 in my project and it seems to work but I have not tested with the latest dataproc

I shall be looking into this over the weekend, but if any of you want to do a headstart and test it out I would really appreciate it

Answer 3 · 2019-03-21T11:56:47.000Z

Hi all,
I have try, WITHOUT success, packing guava library on the jar adding dependency for guava v18, in the

<dependency>
    <groupId>com.google.guava</groupId>
    <artifactId>guava</artifactId>
    <version>18.0</version>
</dependency>

Any other think I could try?
Thanks.
Felix

Answer 4 · 2019-03-23T22:07:47.000Z

This issue is now fixed, please try the latest release and reopen if not the case

Thanks for raising it

Answer 5 · 2019-03-27T14:25:10.000Z

Thanks for your time! but it seems not work for me.
I have update pom file to get new version 0.2.6 how ever the error persist.

BigQueryClient:141 - Executing query SELECT * FROM [business-intelligence-noticias:DS_DEV_minsait_dev.Rel_CanalCaptacion]
2019-03-27 15:12:22 INFO BigQueryClient:164 - Staging dataset datanoticiasdesarrollo:spark_bigquery_staging_eu already exists
2019-03-27 15:12:22 INFO BigQueryClient:144 - Destination table: {datasetId=spark_bigquery_staging_eu, projectId=datanoticiasdesarrollo, tableId=spark_bigquery_20190327141222_1158602716}
2019-03-27 15:12:22 INFO BigQueryClient:200 - Using legacy Sql: true
Exception in thread "main" java.lang.NoSuchMethodError: com.google.common.base.Preconditions.checkArgument(ZLjava/lang/String;Ljava/lang/Object;Ljava/lang/Object;)V
at com.google.cloud.hadoop.io.bigquery.BigQueryStrings.parseTableReference(BigQueryStrings.java:68)
.
I include the maven tree output if it helps.
mvnTres.txt
Thanks in advance
Felix

Answer 6 · 2019-06-03T12:42:42.000Z

yes It still persists, I am also getting this error
Caused by: java.lang.NoSuchMethodError: com.google.common.base.Preconditions.checkArgument(ZLjava/lang/String;Ljava/lang/Object;Ljava/lang/Object;)V
at com.google.cloud.hadoop.io.bigquery.BigQueryStrings.parseTableReference(BigQueryStrings.java:68)
at com.samelamin.spark.bigquery.BigQueryDataFrame.saveAsBigQueryTable(BigQueryDataFrame.scala:40)
at com.samelamin.spark.bigquery.streaming.BigQuerySink.addBatch(BigQuerySink.scala:29)
......
....
...

Using guava v19, spark 2.3 on dataproc.

Answer 7 · 2019-06-12T09:14:00.000Z

I also face the same issue with latest version.

Answer 8 · 2019-06-17T06:53:37.000Z

Similar issue on my side, didnt found work arround yet

Answer 9 · 2019-06-17T09:34:06.000Z

I created a workflow with this dependencies provided by the cluster :
And it solves differents versions issues

jobs:
  - sparkJob:
      jarFileUris:
        - gs://bucket/xxxx-0.1.0-SNAPSHOT.jar
      mainClass: jobs.BQ2GCS
    stepId: step1
placement:
  managedCluster:
    clusterName: xxx
    config:
      gceClusterConfig:
        metadata:
          bigquery-connector-version: 0.13.16
          gcs-connector-version: 1.9.16
    
        zoneUri: europe-west1-b
      initializationActions:
        - executableFile: gs://dataproc-initialization-actions/connectors/connectors.sh
          executionTimeout: 600s
      softwareConfig:
        imageVersion: '1.4'
      workerConfig:
        numInstances: 2

Answer 10 · 2019-06-18T08:40:54.000Z

@dgouyette Thank you it is working.
I could also make it works by forcing "com.google.guava:guava:27.0.1-jre" to be use

Answer 11 · 2019-08-22T09:17:50.000Z

I also face the same issue with latest version.

were you able to fix this? If yes , can you please share the soluion. Thanks

Answer 12 · 2019-08-22T09:18:53.000Z

@dgouyette Thank you it is working.
I could also make it works by forcing "com.google.guava:guava:27.0.1-jre" to be use

This still doesnt resolve the issue for me

Answer 13 · 2019-10-05T23:10:43.000Z

I came across this error when running on a newer version of Dataproc cluster.

To resolve it do the following in your build (mine assembly) with shade rules to get rid of old references to guava

assemblyShadeRules in assembly := Seq(
ShadeRule.rename("com.google.common.**" -> "my_conf.@1").inAll
)

Also see this dated email of mine

https://groups.google.com/forum/#!msg/cloud-dataproc-discuss/OICSgtN2oe8/bhOswni8EAAJ

currently I am using this dependency in my assembly

libraryDependencies += "com.google.guava" % "guava" % "27.0.1-jre"

HTH,

Mich

Answer 14 · 2019-10-06T11:20:13.000Z

Thanks Mich, guava is such a pain. Essentially this is fixed by shading the google common libraries

If the above doesnt work I am happy reopening this ticket