FedJax depends on TensorFlow Federated?
davidrpugh opened this issue · 7 comments
I am helping users install FedJax for use in their federated learning research projects and I noticed that installing FedJax is pulling in TensorFlow Federated (0.17) and TensorFlow (2.3). I don't see either of these listed as dependencies of FedJax so I am trying to understand why they are being pulled in by pip install fedjax.
TensorFlow Federated was a dependency in earlier versions of FedJAX. It is no longer needed at HEAD, which can be installed via pip install --upgrade git+https://github.com/google/{fed,}jax.git. We will soon push out a new release to PyPI (our upcoming release is depending on a new JAX release because of the recent renaming of jax.experimental).
We actually do use TensorFlow in some dataset preprocessing logic for stackoverflow and cifar100, as well as some IO operations here and there, but forgot to list TensorFlow in requirements. Since most part of FedJAX should function without TensorFlow, if not depending on TensorFlow is important to you, there is a way of making things work.
The issue that I am having is that when installing fedjax via pip using the above is that it pulls in TensorFlow 2.3 which requires CUDA 10.1 while Jax itself strongly recommends 11.1+ because of known bugs in CUDA 10.1.
I was able to solve the issue by explicitly pinning the version of TensorFlow required to be 2.6. This allows me to install FedJax, Jax, and TensorFlow together with CUDA 11.2 and things appear to work (although I can claim super thorough testing). Might want to consider putting a lower bound on the TF version in the setup.py file.
Sorry about the inconvenience caused by this.
FedJAX actually only needs the CPU only portion of TensorFlow, which can be installed with pip as tensorflow-cpu. Unfortunately setuptools doesn't allow us to specify a "this or that" dependency requirement, and if we list tensorflow-cpu as one of the requirements, pip will silently overwrite people's existing standard tensorflow installation, which is highly undesirable. Therefore we are moving towards a direction to make tensorflow/tensorflow-cpu an optional dependency so that the user can choose which one to install.
The issue that I am having is that when installing
fedjaxviapipusing the above is that it pulls in TensorFlow 2.3 which requires CUDA 10.1 while Jax itself strongly recommends 11.1+ because of known bugs in CUDA 10.1.
I suspect TF 2.3 was being installed because your pip version was too old. That's why we recommend upgrading pip first before installing FedJAX in our README.
We pushed out v0.0.8 last night. TensorFlow Federated is no longer a dependency, and TensorFlow is now an optional dependency. Installing fedjax via pip will not pull TensorFlow, and an exception with instructions for installing TensorFlow will be printed when a feature requiring TensorFlow is being used.
Please feel free to try out the new release, and let us know if you still run into issues.