outdated pyspark installs
Closed this issue · 2 comments
Notebook: https://github.com/IBM/claimed/blob/master/coursera_ds/assignment1.2.ipynb (and later assignments in the coursera_ds directory) uses !pip install pyspark==2.4.5 under the assumption that the user set up an environment with Python 3.6 (https://github.com/IBM/claimed/wiki/Watson-Studio-Setup). Python 3.6 is not available... the only Python environment that I'm able to create is 3.9, which is incompatible with 2.4.5. It's not clear whether I'm okay commenting out the version on pyspark and using the current version (3.3). Given I don't know pyspark (hence, why I'm taking the coursera class), I don't know if things down the road will break by switching to 3.3. At any rate, the notebook should not force the installation of an outdated pyspark version if such a version is not compatible with the Python environments that are available.
dear @jamesnordlund thank you very much for your feedback and you are completely right, let me take care of it
@jamesnordlund fixed in 3e4f98c