1.5.0 spark shaded protobuf version conflict with application
hannin opened this issue · 2 comments
Expected behavior
My application uses protobuf also with version higher than 3.19.x
with org.apache.sedona:sedona-spark-shaded-3.0_2.12:1.5.0
, protobuf generated files makes error like
if (!com.google.protobuf.GeneratedMessageV3.isStringEmpty(osVersion_)) {
^
symbol: method isStringEmpty(java.lang.Object)
location: class com.google.protobuf.GeneratedMessageV3
It's because protoc version and protobuf-java lib version not matched
Actual behavior
with org.apache.sedona:sedona-spark-shaded-3.0_2.12:1.5.0, custom protobuf-java version builds
Steps to reproduce the problem
I made sample project for reproduce https://oss.navercorp.com/tak/spark-sedona-test
./gradlew build
failed there
Settings
Sedona version = 1.5.0
Apache Spark version = not explicit
Apache Flink version = no
API type = Scala
Scala version = 2.12
JRE version = 1.8
Python version = no
Environment = build phase
with shaded jar, there is relocate function to fix this (ref https://maven.apache.org/plugins/maven-shade-plugin/examples/class-relocation.html)
@hannin Since you are using gradle, you can use our unshaded jar: https://sedona.apache.org/1.5.0/setup/maven-coordinates/#use-sedona-unshaded-jars
Here is an example Scala project that uses the unshaded jar. You can exclude protobuf-java from the unshaded dependency. https://github.com/apache/sedona/blob/master/examples/spark-sql/pom.xml
oh there was documented solution already.. thank you