Goosang-Yu/genet

GenET v0.15.0 installation problom on Apple Silicon (M1 / M2)

Closed this issue · 6 comments

Goosang-Yu commented

I gave it a try but having some sort of installation issue, sorry! I am guessing related to #86. Here is what I tried:

In Terminal:

conda create -n deepprime2
conda activate deepprime2
conda config --env --set subdir osx-64
conda install python=3.10
pip install genet

It said it worked.

Then in a Jupyter Notebook:

from genet.predict import DeepPrime

This runs forever... e.g. print('hey') works so I do not think it is a Python/Jupyter issue. I do not think it's the same as #84 as here it's getting stuck at import phase. Is it still missing some dependency?

Originally posted by @francoiskroll in #89 (comment)

Goosang-Yu commented

@francoiskroll

Thanks for checking!
It seems like the issues we're encountering are likely due to TensorFlow.

After some search, it appears that there is a separate TensorFlow version for ARM-powered (M1/M2) MacOS.
Could you uninstall TensorFlow in your 'deepprime2' environment, reinstall the MacOS version, and then check the import of DeepPrime?

pip uninstall tensorflow
pip install tensorflow-macos==2.9

If that doesn't resolve the issue, could you install some additional plug-ins and recheck, please?

Install dependencies for tensorflow-macos

conda install -c apple tensorflow-deps

Install plug-in for GPU support of tensorflow-macos

pip install tensorflow-metal==0.5.0

Thanks!

Related references:

TensorFlow Installation on Your M2 Chip MacBook
Installing Tensorflow on Apple Silicon
TensorFlow Installation on Mac M1/M2(Apple Silicon) Chip | Quick Setup Guide

francoiskroll commented

Ok, so my understanding is as I set the conda environment with conda config --env --set subdir osx-64, I should be installing the osx-64 versions of packages, not arm64. I set it this way because e.g. viennarna does not have an arm64 version.

But yes, the ideal solution would be to make genet work on arm64 without any loophole if possible.

pip uninstall tensorflow
pip install tensorflow-macos==2.9

ERROR: Could not find a version that satisfies the requirement tensorflow-macos==2.9 (from versions: none)
ERROR: No matching distribution found for tensorflow-macos==2.9

I think because the environment is set as osx-64 it doesn't find the arm64 install?

If I simply run

pip install tensorflow

I get tensorflow 2.16.1, together with error:

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
genet 0.15.0 requires protobuf<=3.20.3, but you have protobuf 4.25.3 which is incompatible.
genet 0.15.0 requires tensorflow<2.10.0, but you have tensorflow 2.16.1 which is incompatible.

And from genet.predict import DeepPrime still runs forever.

I tried installing the other dependencies, just in case:

conda install -c apple tensorflow-deps

PackagesNotFoundError: The following packages are not available from current channels.

pip install tensorflow-metal==0.5.0

ERROR: Could not find a version that satisfies the requirement tensorflow-metal==0.5.0 (from versions: none)
ERROR: No matching distribution found for tensorflow-metal==0.5.0.

I think same reason as before, it's only looking for osx-64 packages.

Ok, how about I try a new environment sticking to your installation instructions and leave everything arm64?

conda create -n deepprime3 python=3.10
conda activate deepprime3
pip install genet

ERROR: Cannot install genet==0.11.0, genet==0.12.0, genet==0.13.0, genet==0.13.1, genet==0.13.2, genet==0.13.3, genet==0.13.4, genet==0.13.5, genet==0.13.6, genet==0.13.7, genet==0.14.0, genet==0.14.1 and genet==0.15.0 because these package versions have conflicting dependencies.

The conflict is caused by:
genet 0.15.0 depends on tensorflow<2.10.0
genet 0.14.1 depends on tensorflow<2.10.0
genet 0.14.0 depends on tensorflow<2.10.0
genet 0.13.7 depends on tensorflow<2.10.0
genet 0.13.6 depends on tensorflow<2.10.0
genet 0.13.5 depends on tensorflow<2.10.0
genet 0.13.4 depends on tensorflow<2.10.0
genet 0.13.3 depends on tensorflow<2.10.0
genet 0.13.2 depends on tensorflow<2.10.0
genet 0.13.1 depends on tensorflow<2.10.0
genet 0.13.0 depends on tensorflow<2.10.0
genet 0.12.0 depends on tensorflow<2.10.0
genet 0.11.0 depends on tensorflow<2.10.0

To fix this you could try to:

  1. loosen the range of package versions you've specified
  2. remove package versions to allow pip attempt to solve the dependency conflict

ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-dependency-conflicts

So now trying again pip install tensorflow-macos==2.9. This time it works, so running again pip install genet.

[same error as above] The conflict is caused by:
genet 0.15.0 depends on tensorflow<2.10.0 [...]

Thing is, pip install tensorflow-macos==2.9 gives me "tensorflow-macos", not "tensorflow", so think genet doesn't see it?

  • 👍1
Goosang-Yu commented

Thanks for sharing the test results!
It seems like your opinion about tensorflow and tensorflow-macos being distinguished differently in pip is correct.

I'm planning to test and release by specifying different installations based on the platform_system in the package dependency (pyproject.toml/dependencies).

dependencies = [
    'pandas',
    'regex',
    'biopython',
    'torch',
    'tensorflow < 2.10.0 ; platform_system != "Darwin"',
    'tensorflow-macos < 2.10.0 ; platform_system == "Darwin"',
    'protobuf <= 3.20.3',
    'silence_tensorflow',
    'pyarrow',
    'fastparquet',
    'tqdm',
    'editdistance',
    'viennarna',
    ]

While I don't have a silicon MacBook for immediate testing, I'll seek help from friends around me who have one to test as soon as possible and let you know the results.

Thanks again for your help!

francoiskroll commented

No problem. I'll continue using genet 0.14.0 for now and delete those environments (Jupyter loves switching the Python & kernel to them for some reason), but let me know if I can help further!

Goosang-Yu commented

GenET has been updated to install proper tensorflow version on MacOS.

This change will be applied starting from v0.15.1 (#93). I've verified that it installs correctly on an M2 MacBook Air (python=3.10 without changing conda config setting and installing any other plugins).

conda create -n genet python=3.10
pip install genet

If you have time, I would really appreciate it if you could check if it installs without any issues on your computer as well.

Thank you!

francoiskroll commented

It works 🎉! Really nice job. Thanks for the effort.

  • 😄1