Problem with pip installation

Question

Problem with pip installation

ShambaMondal opened this issue 2 years ago · 11 comments

Hi,
Thank you for creating MELD. I tried to install MELD with pip v23.0 within a dedicated conda environment (anaconda on ubuntu) for the package and got the error mentioned below.

$ pip install meld
Collecting meld
Using cached meld-1.0.0-py3-none-any.whl (26 kB)
Collecting numpy>=1.14.0
Downloading numpy-1.24.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (17.3 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 17.3/17.3 MB 50.4 MB/s eta 0:00:00
Collecting scipy>=1.1.0
Downloading scipy-1.10.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (34.1 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 34.1/34.1 MB 37.5 MB/s eta 0:00:00
Collecting graphtools>=1.5.0
Downloading graphtools-1.5.3-py3-none-any.whl (45 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 45.4/45.4 kB 9.9 MB/s eta 0:00:00
Collecting pandas>=0.25
Downloading pandas-1.5.3-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (12.0 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 12.0/12.0 MB 45.5 MB/s eta 0:00:00
Collecting scprep>=1.0
Downloading scprep-1.2.2-py3-none-any.whl (93 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 93.8/93.8 kB 10.6 MB/s eta 0:00:00
Collecting pygsp
Using cached PyGSP-0.5.1-py2.py3-none-any.whl (1.8 MB)
Collecting sklearn
Downloading sklearn-0.0.post1.tar.gz (3.6 kB)
Preparing metadata (setup.py) ... error
error: subprocess-exited-with-error

× python setup.py egg_info did not run successfully.
│ exit code: 1
╰─> [18 lines of output]
The 'sklearn' PyPI package is deprecated, use 'scikit-learn'
rather than 'sklearn' for pip commands.

  Here is how to fix this error in the main use cases:
  - use 'pip install scikit-learn' rather than 'pip install sklearn'
  - replace 'sklearn' by 'scikit-learn' in your pip requirements files
    (requirements.txt, setup.py, setup.cfg, Pipfile, etc ...)
  - if the 'sklearn' package is used by one of your dependencies,
    it would be great if you take some time to track which package uses
    'sklearn' instead of 'scikit-learn' and report it to their issue tracker
  - as a last resort, set the environment variable
    SKLEARN_ALLOW_DEPRECATED_SKLEARN_PACKAGE_INSTALL=True to avoid this error
  
  More information is available at
  https://github.com/scikit-learn/sklearn-pypi-package
  
  If the previous advice does not cover your use case, feel free to report it at
  https://github.com/scikit-learn/sklearn-pypi-package/issues/new
  [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

Then I tried to install it with pip v22.2.2 at the default "base" environment (anaconda). This time it was successfully installed, but it downgraded a few packages (including pandas) and even installed deprecated packages, thus modifying the global environment.

$ pip install meld
Collecting meld
Using cached meld-1.0.0-py3-none-any.whl (26 kB)
Collecting scprep>=1.0
Using cached scprep-1.2.2-py3-none-any.whl (93 kB)
Requirement already satisfied: scipy>=1.1.0 in /home/sam/softwares/anaconda3/lib/python3.8/site-packages (from meld) (1.9.1)
Requirement already satisfied: pandas>=0.25 in /home/sam/softwares/anaconda3/lib/python3.8/site-packages (from meld) (1.5.0)
Collecting sklearn
Using cached sklearn-0.0.post1.tar.gz (3.6 kB)
Preparing metadata (setup.py) ... done
Collecting graphtools>=1.5.0
Using cached graphtools-1.5.3-py3-none-any.whl (45 kB)
Collecting pygsp
Using cached PyGSP-0.5.1-py2.py3-none-any.whl (1.8 MB)
Requirement already satisfied: numpy>=1.14.0 in /home/sam/softwares/anaconda3/lib/python3.8/site-packages (from meld) (1.23.1)
Collecting tasklogger>=1.0
Using cached tasklogger-1.2.0-py3-none-any.whl (14 kB)
Collecting Deprecated
Using cached Deprecated-1.2.13-py2.py3-none-any.whl (9.6 kB)
Requirement already satisfied: future in /home/sam/softwares/anaconda3/lib/python3.8/site-packages (from graphtools>=1.5.0->meld) (0.18.2)
Requirement already satisfied: scikit-learn>=0.20.0 in /home/sam/softwares/anaconda3/lib/python3.8/site-packages (from graphtools>=1.5.0->meld) (1.1.2)
Requirement already satisfied: python-dateutil>=2.8.1 in /home/sam/softwares/anaconda3/lib/python3.8/site-packages (from pandas>=0.25->meld) (2.8.2)
Requirement already satisfied: pytz>=2020.1 in /home/sam/softwares/anaconda3/lib/python3.8/site-packages (from pandas>=0.25->meld) (2022.4)
Requirement already satisfied: decorator>=4.3.0 in /home/sam/softwares/anaconda3/lib/python3.8/site-packages (from scprep>=1.0->meld) (5.1.1)
Collecting pandas>=0.25
Using cached pandas-1.3.5-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.5 MB)
Requirement already satisfied: packaging in /home/sam/softwares/anaconda3/lib/python3.8/site-packages (from scprep>=1.0->meld) (21.3)
Requirement already satisfied: six>=1.5 in /home/sam/softwares/anaconda3/lib/python3.8/site-packages (from python-dateutil>=2.8.1->pandas>=0.25->meld) (1.16.0)
Requirement already satisfied: threadpoolctl>=2.0.0 in /home/sam/softwares/anaconda3/lib/python3.8/site-packages (from scikit-learn>=0.20.0->graphtools>=1.5.0->meld) (3.1.0)
Requirement already satisfied: joblib>=1.0.0 in /home/sam/softwares/anaconda3/lib/python3.8/site-packages (from scikit-learn>=0.20.0->graphtools>=1.5.0->meld) (1.2.0)
Requirement already satisfied: wrapt<2,>=1.10 in /home/sam/softwares/anaconda3/lib/python3.8/site-packages (from Deprecated->graphtools>=1.5.0->meld) (1.14.1)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /home/sam/softwares/anaconda3/lib/python3.8/site-packages (from packaging->scprep>=1.0->meld) (3.0.9)
Building wheels for collected packages: sklearn
Building wheel for sklearn (setup.py) ... done
Created wheel for sklearn: filename=sklearn-0.0.post1-py3-none-any.whl size=2936 sha256=36d93cd3e38933b3d34e0cef8347f52a73dbe98b345ae1999843b4094baa4cb9
Stored in directory: /home/sam/.cache/pip/wheels/14/25/f7/1cc0956978ae479e75140219088deb7a36f60459df242b1a72
Successfully built sklearn
Installing collected packages: sklearn, Deprecated, tasklogger, pygsp, pandas, scprep, graphtools, meld
Attempting uninstall: pandas
Found existing installation: pandas 1.5.0
Uninstalling pandas-1.5.0:
Successfully uninstalled pandas-1.5.0
Successfully installed Deprecated-1.2.13 graphtools-1.5.3 meld-1.0.0 pandas-1.3.5 pygsp-0.5.1 scprep-1.2.2 sklearn-0.0.post1 tasklogger-1.2.0

Could you please suggest a solution?

Thanks!

dsm-72 commented a year ago

@atong01

Answer 1 · 2023-02-04T19:47:17.000Z

This is due to upstream sklearn install deprecation. See here: https://github.com/scikit-learn/sklearn-pypi-package

This issue should resolve itself soon as upstream packages update their dependencies. If its a problem for you, either the solution you identified or setting SKLEARN_ALLOW_DEPRECATED_SKLEARN_PACKAGE_INSTALL=True should fix this problem.

Answer 2 · 2023-02-07T18:46:34.000Z

I'm not sure where this is coming from (there aren't many dependencies of MELD, and they all reference scikit-learn.

I also cannot reproduce. @ShambaMondal are you still having this issue? I'm wondering if it has something to do with the conda repository you're using.

Answer 3 · 2023-02-07T19:02:35.000Z

Looking at pipdeptree this is actually a problem with meld. @dburkhardt we need to update the release. meld==1.0.0 depends on sklearn directly. Looks like main go rid of this but we never released an update to pip.

Answer 4 · 2023-02-07T21:37:56.000Z

Hmm, the PyPI deploy workflow is supposed to deploy to PyPI, I wonder why it didnt. The current version.py says the version is 1.0.1

Answer 5 · 2023-02-07T21:45:08.000Z

Note that I pushed tag 1.0.1 to try to get a deploy today. Looks like the action is out of date see PR and failing run below:

https://github.com/KrishnaswamyLab/MELD/actions/runs/4118471691

Answer 6 · 2023-02-07T21:53:14.000Z

Also note that the action succeeds but does not push to pypi if there is not a tag. To get it to deploy to pypi you need to push a new tag.

Answer 7 · 2023-02-07T23:31:37.000Z

Hi, thanks for getting back to the issue. After I reported the issue above, I did the following things:

I updated conda and anaconda. All the packages were updated.
$ conda update conda
$ conda update anaconda
I tried installing MELD with pip again (“pip install meld”; new pip version this time, v 23.0, after updating the environment) at the "base" environment. This time the installation process was much cleaner than the last time, most possibly because we had updated/installed majority of the dependencies via conda/anaconda update. But pandas again got downgraded from v1.5.3 to 1.3.5. This was the last line of the output: "Successfully installed meld-1.0.0 pandas-1.3.5"
Next, I checked if recreating the environment "env_meld" with updated conda/anaconda would make any difference. It made a difference indeed! This time meld was successfully installed within the environment. However, because pip was used directly to install meld within the otherwise empty env_meld environment, the versions of the dependencies were now different within env_meld than in the base environment.

(base) $ conda env remove --name env_meld (<-- removing older env)
(base) $ conda create --name env_meld
(base) $ conda install -n env_meld pip
(base) $ conda activate env_meld
(env_meld) $ pip --version
pip 23.0 from <pip_PATH>/pip (python 3.11)
(env_meld) $ pip install meld

last line of the output: "Successfully installed Deprecated-1.2.13 decorator-5.1.1 future-0.18.3 graphtools-1.5.3 joblib-1.2.0 meld-1.0.0 numpy-1.24.1 packaging-23.0 pandas-1.3.5 pygsp-0.5.1 python-dateutil-2.8.2 pytz-2022.7.1 scikit-learn-1.2.1 scipy-1.10.0 scprep-1.2.2 six-1.16.0 sklearn-0.0.post1 tasklogger-1.2.0 threadpoolctl-3.1.0 wrapt-1.14.1"

Finally, I removed all existing MELD installations from "base" and "env_meld" environment (removed "env_meld" itself as well). I wanted meld, magic, phate, batchglm and diffxpy to be installed in the same environment. So I created a new environment with as many dependencies as possible to be installed with conda, then installed meld, magic, phate, batchglm and diffxpy with pip within the environment. The version meld-1.0.0 was installed (see below).

(base) $ conda create -n env_meld_diffxpy python pip pandas numpy graphtools phate scprep cmocean scikit-learn scipy pygsp seaborn tensorflow tensorflow-probability

base) $ conda activate env_meld_diffxpy

(env_meld_diffxpy) $ pip install meld
last line of the output: "Successfully installed meld-1.0.0 pandas-1.3.5 sklearn-0.0.post1"

(env_meld_diffxpy) $ pip install magic-impute
last line of the output: "Successfully installed magic-impute-3.0.0"

(env_meld_diffxpy) $ pip install phate
phate version installed: 1.0.10

(env_meld_diffxpy) $ pip install batchglm
last line of the output: "Successfully installed anndata-0.8.0 batchglm-0.7.4 dask-2023.1.1 fsspec-2023.1.0 llvmlite-0.39.1 locket-1.0.0 natsort-8.2.0 numba-0.56.4 numpy-1.23.5 partd-1.3.0 pyyaml-6.0 sparse-0.13.0 toolz-0.12.0"

(env_meld_diffxpy) $ pip install diffxpy
last line of the output: "Successfully installed diffxpy-0.7.4"

After these installations, I could run the MELD tutorial without any problem. A few warnings about dated usage of older functions and parameters in the tutorial notebook were reported, such as, for sns.kdeplot():

"The bw parameter is deprecated in favor of bw_method and bw_adjust.
Setting bw_method=0.2, but please see the docs for the new parameters
and update your code. This will become an error in seaborn v0.13.0."

and "shade is now deprecated in favor of fill; setting fill=True.
This will become an error in seaborn v0.14.0; please update your code."

Thanks,
Sam

Answer 8 · 2023-02-08T21:04:45.000Z

@ShambaMondal the other thing you could do is just install from source:

git clone the repo
cd into the directory you just downloaded
pip install .
should work

Answer 9 · 2023-04-20T16:12:33.000Z

@dburkhardt can we get a bump on this?

Answer 10 · 2023-07-20T12:42:04.000Z

Fixed in v1.0.2