What is now the best Spark Jupyter Notebook image to use that retains Scala support?
deanwampler opened this issue · 0 comments
Previously, the run.sh and run.bat scripts just used jupyter/all-spark-notebook:latest for the Docker image. However, Scala support was removed in July 2022 (https://github.com/jupyter/docker-stacks, commit: 5048b022466a8c7d4ecff149396e6bf35babb9c8).
I will shortly release an update to the run.* scripts and the README instructions that hard-code the last image that appears to retain Scala support jupyter/all-spark-notebook:spark-3.2.0 (which actually appears to support 3.2.1). Since Spark 3.3.0 recently came out, it won't be possible to keep the tutorial up to date with Spark releases using jupyter/all-spark-notebook.
While the purpose of this tutorial isn't to teach Spark, per se, it would be nice to keep it current with Spark releases. Therefore, this issue is to determine if there is an alternative to jupyter/all-spark-notebook that still maintains Scala support.