Cannot Install from source and Anaconda

[himaghna]

Issue Description

Could not install beanmachine from GitHub code using the recommended Anaconda installation

Steps to Reproduce

git clone https://github.com/facebookresearch/beanmachine.git
cd beanmachine
conda create -n {env name} python=3.8; conda activate {env name}
conda install -c conda-forge boost-cpp eigen=3.4.0
python -m pip install .

raises the error

ERROR: Could not find a version that satisfies the requirement functorch>=0.1.0 (from beanmachine) (from versions: none)
ERROR: No matching distribution found for functorch>=0.1.0

Expected Behavior

Successful installation

System Info

• MacOS 12.2.1 (M1 chip)
• PyTorch Version 1.10.0
• Python version 3.8.13

Additional Context

Complete trace

Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Collecting tqdm>=4.46.0
  Using cached tqdm-4.64.0-py2.py3-none-any.whl (78 kB)
Collecting astor>=0.7.1
  Using cached astor-0.8.1-py2.py3-none-any.whl (27 kB)
Collecting statsmodels>=0.12.0
  Using cached statsmodels-0.13.2-cp38-cp38-macosx_11_0_arm64.whl (9.1 MB)
Collecting black==21.4b2
  Using cached black-21.4b2-py3-none-any.whl (130 kB)
Collecting plotly>=2.2.1
  Using cached plotly-5.7.0-py2.py3-none-any.whl (28.8 MB)
Collecting xarray>=0.16.0
  Using cached xarray-2022.3.0-py3-none-any.whl (870 kB)
Collecting botorch>=0.5.1
  Using cached botorch-0.6.4-py3-none-any.whl (363 kB)
Collecting gpytorch>=1.3.0
  Using cached gpytorch-1.6.0-py2.py3-none-any.whl
Collecting graphviz>=0.17
  Using cached graphviz-0.20-py3-none-any.whl (46 kB)
Collecting pandas>=0.24.2
  Using cached pandas-1.4.2-cp38-cp38-macosx_11_0_arm64.whl (9.9 MB)
Collecting scipy>=0.16
  Using cached scipy-1.8.0-cp38-cp38-macosx_12_0_arm64.whl (28.5 MB)
ERROR: Could not find a version that satisfies the requirement functorch>=0.1.0 (from beanmachine) (from versions: none)
ERROR: No matching distribution found for functorch>=0.1.0

[zaxtax]

@horizon-blue any ideas what could be causing this?

[horizon-blue]

Hi @himaghna, seems like this particular error is caused by the functorch dependency that we recently introduced, which doesn't have a prebuild binary for M1 macbook. You can try manually install functorch from source with the following command (you can find more discussion around the issue here: pytorch/functorch#608 (comment)):

pip install --upgrade git+https://github.com/pytorch/functorch@v0.1.0

I will also update the setup script for Bean Machine to make the functorch module optional. Thanks for reporting the installation issue :).

[himaghna]

Hi @himaghna, seems like this particular error is caused by the functorch dependency that we recently introduced, which doesn't have a prebuild binary for M1 macbook. You can try manually install functorch from source with the following command (you can find more discussion around the issue here: pytorch/functorch#608 (comment)):

pip install --upgrade git+https://github.com/pytorch/functorch@v0.1.0

I will also update the setup script for Bean Machine to make the functorch module optional. Thanks for reporting the installation issue :).

Thank you! This builds beanmachine and all imports work. However, the build does not pass the tests (not sure if this should be a separate issue).

Steps to Reproduce

python -m pip install "beanmachine[test]"
pytest src

fails with

==================================== ERRORS ====================================
________________________ ERROR collecting test session _________________________
src/beanmachine/ppl/conftest.py:6: in <module>
    import beanmachine.ppl as bm
../miniforge3/envs/bean_env/lib/python3.8/site-packages/beanmachine/ppl/__init__.py:9: in <module>
    from .diagnostics import Diagnostics
../miniforge3/envs/bean_env/lib/python3.8/site-packages/beanmachine/ppl/diagnostics/__init__.py:11: in <module>
    from beanmachine.ppl.diagnostics.diagnostics import Diagnostics
../miniforge3/envs/bean_env/lib/python3.8/site-packages/beanmachine/ppl/diagnostics/diagnostics.py:14: in <module>
    from beanmachine.ppl.inference.monte_carlo_samples import MonteCarloSamples
../miniforge3/envs/bean_env/lib/python3.8/site-packages/beanmachine/ppl/inference/__init__.py:6: in <module>
    from beanmachine.ppl.inference.bmg_inference import BMGInference
../miniforge3/envs/bean_env/lib/python3.8/site-packages/beanmachine/ppl/inference/bmg_inference.py:22: in <module>
    from beanmachine.ppl.compiler.runtime import BMGRuntime
../miniforge3/envs/bean_env/lib/python3.8/site-packages/beanmachine/ppl/compiler/runtime.py:63: in <module>
    from beanmachine.ppl.compiler.special_function_caller import (
../miniforge3/envs/bean_env/lib/python3.8/site-packages/beanmachine/ppl/compiler/special_function_caller.py:16: in <module>
    from beanmachine.ppl.compiler.beanstalk_common import allowed_functions
../miniforge3/envs/bean_env/lib/python3.8/site-packages/beanmachine/ppl/compiler/beanstalk_common.py:6: in <module>
    from beanmachine.ppl.model.statistical_model import random_variable, functional
../miniforge3/envs/bean_env/lib/python3.8/site-packages/beanmachine/ppl/model/__init__.py:7: in <module>
    from beanmachine.ppl.model.statistical_model import (
../miniforge3/envs/bean_env/lib/python3.8/site-packages/beanmachine/ppl/model/statistical_model.py:11: in <module>
    from beanmachine.ppl.experimental.vi.variational_world import VariationalWorld
E   ModuleNotFoundError: No module named 'beanmachine.ppl.experimental.vi'
=========================== short test summary info ============================
ERROR  - ModuleNotFoundError: No module named 'beanmachine.ppl.experimental.vi'
!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!
=============================== 1 error in 3.62s ===============================

[horizon-blue]

Ah, that's caused by the lack of an __init__.py file under src/beanmachine/ppl/experimenta/vi/. We are in the middle of updating the VI framework, and the init file gets removed accidentally as part of the migration :). This will be fixed soon.

In the mean time, you can try install Bean Machine in editable mode (with pip install -e .) to see if that resolves the issue

[himaghna]

Ah, I see! Installing in editing mode resolves the issue but the tests fail differently. Thanks for the response! I am leaving the error trace here but I can open a new issue if this is unrelated to this one.

Steps to Reproduce

pip install -e .
pytest src

fails with error

collected 0 items / 1 error                                                      

===================================== ERRORS =====================================
_________________________ ERROR collecting test session __________________________
src/beanmachine/ppl/conftest.py:6: in <module>
    import beanmachine.ppl as bm
src/beanmachine/ppl/__init__.py:9: in <module>
    from .diagnostics import Diagnostics
src/beanmachine/ppl/diagnostics/__init__.py:11: in <module>
    from beanmachine.ppl.diagnostics.diagnostics import Diagnostics
src/beanmachine/ppl/diagnostics/diagnostics.py:14: in <module>
    from beanmachine.ppl.inference.monte_carlo_samples import MonteCarloSamples
src/beanmachine/ppl/inference/__init__.py:6: in <module>
    from beanmachine.ppl.inference.bmg_inference import BMGInference
src/beanmachine/ppl/inference/bmg_inference.py:23: in <module>
    from beanmachine.ppl.inference.monte_carlo_samples import MonteCarloSamples
src/beanmachine/ppl/inference/monte_carlo_samples.py:8: in <module>
    import arviz as az
../miniforge3/envs/bean_env/lib/python3.8/site-packages/arviz/__init__.py:32: in <module>
    from .data import *
../miniforge3/envs/bean_env/lib/python3.8/site-packages/arviz/data/__init__.py:3: in <module>
    from .converters import convert_to_dataset, convert_to_inference_data
../miniforge3/envs/bean_env/lib/python3.8/site-packages/arviz/data/converters.py:6: in <module>
    from .inference_data import InferenceData
../miniforge3/envs/bean_env/lib/python3.8/site-packages/arviz/data/inference_data.py:27: in <module>
    import netCDF4 as nc
../miniforge3/envs/bean_env/lib/python3.8/site-packages/netCDF4/__init__.py:3: in <module>
    from ._netCDF4 import *
E   ImportError: dlopen(/Users/himaghnabhattacharjee/miniforge3/envs/bean_env/lib/python3.8/site-packages/netCDF4/_netCDF4.cpython-38-darwin.so, 0x0002): symbol not found in flat namespace '_H5get_libversion'
============================ short test summary info =============================
ERROR  - ImportError: dlopen(/Users/himaghnabhattacharjee/miniforge3/envs/bean_...
!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!!
================================ 1 error in 3.81s ================================

[horizon-blue]

Looks like this is an issue coming from the netCDF4 package this time. Could you try installing a MacOS ARM64 netCDF4 wheel from here (Unidata/netcdf4-python#1148 (comment)) and see if that works for you? If the issue still persist, you might have to report to the netCDF4 folks there to see if they can help resolving the issue :)

[himaghna]

Thanks @horizon-blue !! That seems to be the way forward. :)

[jpchen]

Resolving the issue, feel free to reopen if you run into anything else.