awesome-kyuubi/hadoop-testing

Hudi bundle jar of spark 3.5 cannot be found

Opened this issue · 4 comments

download error:

downloading hudi-spark3.5-bundle_2.12-0.14.1.jar ...
--2024-05-17 13:58:30--  https://github.com/yanghua/incubator-hudi/releases/download/release-0.14.1/hudi-spark3.5-bundle_2.12-0.14.1.jar
Resolving github.com (github.com)... 20.205.243.166
Connecting to github.com (github.com)|20.205.243.166|:443... connected.
HTTP request sent, awaiting response... 404 Not Found
2024-05-17 13:58:31 ERROR 404: Not Found.

cc @yanghua

Hudi 0.15.0 is released with pre-built bundle jars that support Spark 3.5 and Scala 2.13, not sure about Hadoop 3

download error:

downloading hudi-spark3.5-bundle_2.12-0.14.1.jar ...
--2024-05-17 13:58:30--  https://github.com/yanghua/incubator-hudi/releases/download/release-0.14.1/hudi-spark3.5-bundle_2.12-0.14.1.jar
Resolving github.com (github.com)... 20.205.243.166
Connecting to github.com (github.com)|20.205.243.166|:443... connected.
HTTP request sent, awaiting response... 404 Not Found
2024-05-17 13:58:31 ERROR 404: Not Found.

cc @yanghua

It seems that you changed the spark version to 3.5. While I only provided 3.4.
This personal binary package was tried to resolve the conflict from the dependency of hbase. Not sure if the issue has been resolved in hudi 0.15.

@yanghua not sure about Hudi 0.15, but Spark 3.5 drops support for Hadoop2

@yanghua not sure about Hudi 0.15, but Spark 3.5 drops support for Hadoop2

Got it, will try to verify if the official hudi bundle for spark 3.5 works fine.