aws-samples/spark-on-aws-lambda

Iceberg error

Closed this issue · 3 comments

Trying to use Iceberg format gives me these errors:

Missing org.apache.iceberg.aws.glue.GlueCatalog [java.lang.NoClassDefFoundError: software/amazon/awssdk/services/glue/model/EntityNotFoundException]

pyspark.sql.utils.IllegalArgumentException: Cannot initialize Catalog implementation org.apache.iceberg.aws.glue.GlueCatalog: Cannot find constructor for interface org.apache.iceberg.catalog.Catalog

Did build the docker image with Framework ICEBERG and pyspark 3.2.0.

Is this generally not working or am I missing something?

Yes, we have resolution for this issues. Resolving it in PR. Use version 3.3.2.12 will resolve

@reinhardt1kws we have resolution for this issues. Resolving it in PR. Use version 3.3_2.12 will resolve

ARG ICEBERG_FRAMEWORK_VERSION=3.3_2.12

Closing the issue, changed the version of Iceberg