ABRiS Version 3.2.2 with Spark 3.2.1 Throws Error
Closed this issue · 2 comments
Hello,
I have a custom application that I'm running on EMR cluster. The application is using ABRiS Version 3.2.2. It works great with Spark version 3.1.2. However, when I use the application with Spark 3.2.1 it throws the following error:
java.lang.AbstractMethodError: Method za/co/absa/abris/avro/sql/AvroDataToCatalyst.child()Lorg/apache/spark/sql/catalyst/trees/TreeNode; is abstract
Upgrading to latest ABRiS version is not an option as this point. Can you please provide some guidance on how to use ABRiS version 3.2.2 with Spark 3.2.1?
Thank you
Hello,
Spark 3.2.1 is supported by Abris version 6.2.0+
see: https://github.com/AbsaOSS/ABRiS#supported-versions
Upgrading to latest ABRiS version is not an option as this point. Can you please provide some guidance on how to use ABRiS version 3.2.2 with Spark 3.2.1?
Oh ok, then the only alternative to upgrading is to fork Abris and fix the compatibility issues yourself.