dsgibbons/shap

Cannot install package: CUDA Error - Add Fall back option to CPU

StatMixedML opened this issue · 5 comments

StatMixedML commented

Dear community,

Thanks for maintaining the shap-package.

Having installed CUDA-11.7 and using pip install git+https://github.com/dsgibbons/shap.git throws the following error.

shap_cuda_error.txt

Can you add a CPU fallback option to the setup.py to allow non-CUDA installation, thanks

dsgibbons commented

Thank you for your interest in this fork.

By CUDA, I assume you mean Cuda Toolkit? I am able to install shap in a fresh virtual environment using pip install git+https://github.com/dsgibbons/shap.git with CPU-only macOS and Ubuntu machines (both with and without cudatoolkit preinstalled). I notice you are using Windows, so I wonder if that is part of the issue. What other dependencies do you have installed before trying pip install git+https://github.com/dsgibbons/shap.git?

If you manage to find a solution to your issue, please feel free to raise a PR!

thatlittleboy commented

Installation of shap without cuda should already be supported.

The error is that setuptools found multiple egg-info directories (the last few lines of your error log). Can you try the installation of your package (xgboostlss) and this shap fork in a fresh virtual environment? Please provide the detailed steps of the commands you ran (starting with the creation of the virtual environment) if the same error persists. With pip install -vv .... as well, if possible.

On a side note, recursion depth limit is being hit because presumably you're installing this in a directory or path with the "xgboost" name, so we never get a chance to turn off cuda building in our setup() call. I think we can fix that, but I don't think that explains why an exception was raised about multiple egg-info..

  • 👍1
StatMixedML commented

Thanks for your help @thatlittleboy, @dsgibbons

In fact, the problem stems from the xgboost/xgboostlss name of the directory I was trying to install your package in. Renaming the directory not containing any xgboost/xgboostlss name and running the following works, even on a Windows machine

conda create -n xgblss_git python=3.9 -y
conda activate xgblss_git 
pip install -r requirements.txt

with the requirements.txt containing the following

git+https://github.com/dsgibbons/shap.git
git+https://github.com/StatMixedML/XGBoostLSS.git
jupyterlab
ipywidgets

Would be great if you could adjust the setup.py.

thatlittleboy commented

@StatMixedML I've pushed up some fix in #51 . It should at least stop the recursive calls, we would appreciate if you could give it a try and see if you still have any installation issues.

  • 👍1
StatMixedML commented

Thanks @thatlittleboy for your fix, it is working.

To test it, I used the following steps

conda create -n xgboostlss_git python=3.9 -y
pip install -r requirements.txt

with the requirements containing

git+https://github.com/StatMixedML/XGBoostLSS.git
git+https://github.com/dsgibbons/shap.git@recursive-setup

So it is now working, when installing the shap package into a directory and environment that contains xgboost/xgboostlss related names. Also, the setup.py() of xgboostlss contains xgboost-related names. This also didn't work previously.

  • ❤️1