NVIDIA/spark-rapids-jni

[BUG] Drop CentOS7 support in 24.06

NvTimLiu opened this issue · 1 comments

Describe the bug
We plan to drop CentOS 7 (which uses glibc 2.17) & Ubuntu18.04(glibc 2.27) in RAPIDS 24.06. This issue documents some of the work items needed to complete the drop in platform support. Refor to :

https://docs.rapids.ai/notices/rsn0037/

rapidsai/build-planning#23

NVIDIA/spark-rapids#10496

The new minimum glibc supported by RAPIDS will become 2.28 (used by Rocky 8), because that is the oldest glibc of any operating system we currently support.

  • CentOS 7 : 2.17
  • Ubuntu18 : 2.27
  • Rocky8 : 2.28
  • Debian10: 2.28
  • Ubuntu20: 2.31
  • Ubuntu22: 2.35

Will need to:

  • Update scripts/Docker of 24.06 for JNI/spark-rapids-JNI to support Rocky8
  • Build 24.06 cuDF JNI/spark-rapids-JNI on Rocky8
  • Aggregate spark-rapids-jni into 24.06 spark-rapids
  • Drop spark-rapids Integration tests against Ubuntu18 or CentOS7
  • Check IT on CSP to makes sure no CentOS7 and Ubuntu18 or below OS

Built out amd64/arm64 spark-rapids-jni jars on Rocky8, running integration tests with spark-rapids to see if any GLIBC issues.