Can tensorflow/keras be made an optional dependency?
jamesmyatt opened this issue · 7 comments
I don't like installing tensorflow and all of its associated bloat paraphernalia unless I really need to, since the dependencies are extensive and some of the constraints are annoying, e.g. numpy <1.19 and h5py <2.11.
Would it be ok to change this library to make tensorflow and keras optional via extras_install? If so, I can probably make a PR, but I thought I'd check first if it'll be accepted.
I think the same goes for "cvxpy" since it's non-trivial to build on Windows, but this is less of a problem for me, since I install it from conda, and the constraints and extra dependencies are less onorous.
>python -m pip install tensorflow
Collecting tensorflow
Using cached tensorflow-2.3.1-cp38-cp38-win_amd64.whl (342.5 MB)
Requirement already satisfied: wheel>=0.26 in c:\tools\miniconda3\envs\tensorflow-test\lib\site-packages (from tensorflow) (0.35.1)
Collecting absl-py>=0.7.0
Using cached absl_py-0.11.0-py3-none-any.whl (127 kB)
Collecting astunparse==1.6.3
Using cached astunparse-1.6.3-py2.py3-none-any.whl (12 kB)
Requirement already satisfied: wheel>=0.26 in c:\tools\miniconda3\envs\tensorflow-test\lib\site-packages (from tensorflow) (0.35.1)
Collecting gast==0.3.3
Using cached gast-0.3.3-py2.py3-none-any.whl (9.7 kB)
Collecting google-pasta>=0.1.8
Using cached google_pasta-0.2.0-py3-none-any.whl (57 kB)
Collecting grpcio>=1.8.6
Downloading grpcio-1.33.2-cp38-cp38-win_amd64.whl (2.7 MB)
|████████████████████████████████| 2.7 MB 3.3 MB/s
Collecting h5py<2.11.0,>=2.10.0
Using cached h5py-2.10.0-cp38-cp38-win_amd64.whl (2.5 MB)
Collecting keras-preprocessing<1.2,>=1.1.1
Using cached Keras_Preprocessing-1.1.2-py2.py3-none-any.whl (42 kB)
Collecting numpy<1.19.0,>=1.16.0
Using cached numpy-1.18.5-cp38-cp38-win_amd64.whl (12.8 MB)
Collecting opt-einsum>=2.3.2
Using cached opt_einsum-3.3.0-py3-none-any.whl (65 kB)
Collecting protobuf>=3.9.2
Downloading protobuf-3.14.0-py2.py3-none-any.whl (173 kB)
|████████████████████████████████| 173 kB 6.8 MB/s
Collecting six>=1.12.0
Using cached six-1.15.0-py2.py3-none-any.whl (10 kB)
Collecting tensorboard<3,>=2.3.0
Using cached tensorboard-2.4.0-py3-none-any.whl (10.6 MB)
Requirement already satisfied: setuptools>=41.0.0 in c:\tools\miniconda3\envs\tensorflow-test\lib\site-packages (from tensorboard<3,>=2.3.0->tensorflow) (49.6.0.post20201009)
Requirement already satisfied: wheel>=0.26 in c:\tools\miniconda3\envs\tensorflow-test\lib\site-packages (from tensorflow) (0.35.1)
Collecting google-auth<2,>=1.6.3
Downloading google_auth-1.23.0-py2.py3-none-any.whl (114 kB)
|████████████████████████████████| 114 kB 6.4 MB/s
Requirement already satisfied: setuptools>=41.0.0 in c:\tools\miniconda3\envs\tensorflow-test\lib\site-packages (from tensorboard<3,>=2.3.0->tensorflow) (49.6.0.post20201009)
Collecting cachetools<5.0,>=2.0.0
Downloading cachetools-4.1.1-py3-none-any.whl (10 kB)
Collecting google-auth-oauthlib<0.5,>=0.4.1
Using cached google_auth_oauthlib-0.4.2-py2.py3-none-any.whl (18 kB)
Collecting markdown>=2.6.8
Using cached Markdown-3.3.3-py3-none-any.whl (96 kB)
Collecting pyasn1-modules>=0.2.1
Downloading pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
|████████████████████████████████| 155 kB ...
Collecting pyasn1<0.5.0,>=0.4.6
Downloading pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
|████████████████████████████████| 77 kB ...
Collecting requests<3,>=2.21.0
Downloading requests-2.25.0-py2.py3-none-any.whl (61 kB)
|████████████████████████████████| 61 kB ...
Requirement already satisfied: certifi>=2017.4.17 in c:\tools\miniconda3\envs\tensorflow-test\lib\site-packages (from requests<3,>=2.21.0->tensorboard<3,>=2.3.0->tensorflow) (2020.11.8)
Collecting chardet<4,>=3.0.2
Using cached chardet-3.0.4-py2.py3-none-any.whl (133 kB)
Collecting idna<3,>=2.5
Using cached idna-2.10-py2.py3-none-any.whl (58 kB)
Collecting requests-oauthlib>=0.7.0
Downloading requests_oauthlib-1.3.0-py2.py3-none-any.whl (23 kB)
Collecting oauthlib>=3.0.0
Downloading oauthlib-3.1.0-py2.py3-none-any.whl (147 kB)
|████████████████████████████████| 147 kB 6.4 MB/s
Collecting rsa<5,>=3.1.4
Downloading rsa-4.6-py3-none-any.whl (47 kB)
|████████████████████████████████| 47 kB 1.3 MB/s
Collecting tensorboard-plugin-wit>=1.6.0
Using cached tensorboard_plugin_wit-1.7.0-py3-none-any.whl (779 kB)
Collecting tensorflow-estimator<2.4.0,>=2.3.0
Using cached tensorflow_estimator-2.3.0-py2.py3-none-any.whl (459 kB)
Collecting termcolor>=1.1.0
Using cached termcolor-1.1.0-py3-none-any.whl
Collecting urllib3<1.27,>=1.21.1
Downloading urllib3-1.26.2-py2.py3-none-any.whl (136 kB)
|████████████████████████████████| 136 kB 6.8 MB/s
Collecting werkzeug>=0.11.15
Downloading Werkzeug-1.0.1-py2.py3-none-any.whl (298 kB)
|████████████████████████████████| 298 kB 6.4 MB/s
Collecting wrapt>=1.11.1
Using cached wrapt-1.12.1-py3-none-any.whl
Installing collected packages: urllib3, pyasn1, idna, chardet, six, rsa, requests, pyasn1-modules, oauthlib, cachetools, requests-oauthlib, google-auth, werkzeug, tensorboard-plugin-wit, protobuf, numpy, markdown, grpcio, google-auth-oauthlib, absl-py, wrapt, termcolor, tensorflow-estimator, tensorboard, opt-einsum, keras-preprocessing, h5py, google-pasta, gast, astunparse, tensorflow
Successfully installed absl-py-0.11.0 astunparse-1.6.3 cachetools-4.1.1 chardet-3.0.4 gast-0.3.3 google-auth-1.23.0 google-auth-oauthlib-0.4.2 google-pasta-0.2.0 grpcio-1.33.2 h5py-2.10.0 idna-2.10 keras-preprocessing-1.1.2 markdown-3.3.3 numpy-1.18.5 oauthlib-3.1.0 opt-einsum-3.3.0 protobuf-3.14.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 requests-2.25.0 requests-oauthlib-1.3.0 rsa-4.6 six-1.15.0 tensorboard-2.4.0 tensorboard-plugin-wit-1.7.0 tensorflow-2.3.1 tensorflow-estimator-2.3.0 termcolor-1.1.0 urllib3-1.26.2 werkzeug-1.0.1 wrapt-1.12.1
Hi James. Broadly, I agree with you. I find tensorflow/keras and CVXPY requirements annoying also.
I don't know the details of how this would work, but I would be very receptive as long as (1) it's obvious how to install the full and partial dependencies in the README, (2) there are good errors for folks trying to use TF and CVXPY algorithms when those are not installed, and (3) it all still works with pip.
Are those reasonable requirements?
Yes. That's exactly what I had in mind.
Have at it! Thanks.
Howdy. Any update on this? Are you still interested?
I am still interested in this, but I haven't had any time to make progress myself. If someone else wants to do it, then that would be great.
For inspiration, Pandas is quite good at handling optional dependencies. See:
- https://pandas.pydata.org/pandas-docs/stable/getting_started/install.html#optional-dependencies
- https://github.com/pandas-dev/pandas/blob/v1.2.0/pandas/core/frame.py#L2293-L2347
- etc
Similarly Dask lists the optional dependencies as extras:
Gotcha, thanks for the resources.
I only wanted to use the IterativeSVD algorithm without dealing with all the dependency headaches so I made https://github.com/AlexsLemonade/fancySVD. It doesn't solve this issue the Right Way, but maybe it'll be useful to someone so I thought I'd mention it.