Help running ML Example

[aliakbarrehman]

Hi, Thank you so much for maintaining this awesome repository. I am having some problems trying to run ML example on a local Kubernetes cluster. My setup is:

• OS: Windows 10 with WSL2

I managed to setup JupyterHub locally and install JupyterFlow. Basic example is working fine, I have installed tensorflow and keras using the requirements.txt in JupyterHub and managed to run the individual .py files from the notebook with %run input.py but when I submit the workflow to Argo I get

Traceback (most recent call last): File "/home/jovyan/jupyterflow/examples/ml-pipeline/input.py", line 1, in <module> from keras.datasets import mnist ModuleNotFoundError: No module named 'keras'

I am working on forking your excellent work to look into if I can extend it to be able to provide the volumes I want to mount in argo for each step during runtime.

Thanks in advance for your help :)

[hongkunyoo]

Hello, @aliakbarrehman.
Thank you for testing out my project!

It seems that the python path is not in sync. There might be many reasons for the error. Please take a look on followings:

1. Is your volume support ReadWriteMany?
2. Is your python path is under /home/jovyan?

1. Is your volume support ReadWriteMany?

The volume on JupyterHub should be able to be mount on Argo Workflow as well. The volume needs to be shared in order to share python packages.

2. Is your python path is under /home/jovyan?

Your python path should be under home directory(/home/jovyan) given that your volume is mount on home directory.

Type which python both in JupyterHub and Argo Workflow.
They should be under /home/jovyan.

For example,

$ which python
# /home/jovyan/venv/bin/python

If one or another path does not exist under /home/jovyan, the python packages will not be shared.

Please let me know if it still does not work.
Once again, thank you for trying out my project!

[aliakbarrehman]

Hi, Thank you so much for your help. Python path was the problem in my case.