Failing to install `scenario` on MacOS

I've read the FAQ
I've read the Support Policy
I've already searched similar issues and discussions
I've already updated to the most recent supported release (either Stable or Nightly)

Description:

I noticed that macOS is not officially supported. I thought it was good to report anyway.

Steps to reproduce

I tried to install it with

pip install scenario

Output can be found below

Additional context

Output

$pip install scenario Collecting scenario Downloading scenario-1.3.1.tar.gz (151 kB) |████████████████████████████████| 151 kB 352 kB/s Installing build dependencies ... done Getting requirements to build wheel ... done Preparing wheel metadata ... done Collecting packaging Using cached packaging-21.2-py3-none-any.whl (40 kB) Collecting pyparsing<3,>=2.0.2 Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB) Building wheels for collected packages: scenario Building wheel for scenario (PEP 517) ... error ERROR: Command errored out with exit status 1: command: /Users/lorenzorapetti/Software/dic-iit/ADAM/adamEnv/bin/python3.9 /Users/lorenzorapetti/Software/dic-iit/ADAM/adamEnv/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py build_wheel /var/folders/v4/8y04pr2912dd1n1pryz743r80000gp/T/tmpb33hqy26 cwd: /private/var/folders/v4/8y04pr2912dd1n1pryz743r80000gp/T/pip-install-d6fi17h8/scenario_93d92748aa99469aba71337237b49d10 Complete output (80 lines): running bdist_wheel running build running build_ext -- The C compiler identification is AppleClang 12.0.5.12050022 -- The CXX compiler identification is AppleClang 12.0.5.12050022 -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Check for working C compiler: /usr/local/opt/ccache/libexec/cc - skipped -- Detecting C compile features -- Detecting C compile features - done -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Check for working CXX compiler: /usr/local/opt/ccache/libexec/c++ - skipped -- Detecting CXX compile features -- Detecting CXX compile features - done -- Looking for pthread.h -- Looking for pthread.h - found -- Performing Test CMAKE_HAVE_LIBC_PTHREAD -- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success -- Found Threads: TRUE -- Found Python3: /Users/lorenzorapetti/Software/dic-iit/ADAM/adamEnv/bin/python3.9 (found version "3.9.6") found components: Interpreter Development Development.Module Development.Embed -- Using Python: /Users/lorenzorapetti/Software/dic-iit/ADAM/adamEnv/bin/python3.9 CMake Error at CMakeLists.txt:205 (message): Found no Ignition distribution for PyPI package

-- Configuring incomplete, errors occurred!
See also "/private/var/folders/v4/8y04pr2912dd1n1pryz743r80000gp/T/pip-install-d6fi17h8/scenario_93d92748aa99469aba71337237b49d10/build/temp.macosx-11-x86_64-3.9_ScenarioCMakeProject/CMakeFiles/CMakeOutput.log".

==> Configuring:
$ cmake -S /private/var/folders/v4/8y04pr2912dd1n1pryz743r80000gp/T/pip-install-d6fi17h8/scenario_93d92748aa99469aba71337237b49d10 -B /private/var/folders/v4/8y04pr2912dd1n1pryz743r80000gp/T/pip-install-d6fi17h8/scenario_93d92748aa99469aba71337237b49d10/build/temp.macosx-11-x86_64-3.9_ScenarioCMakeProject -GNinja -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX:PATH=/private/var/folders/v4/8y04pr2912dd1n1pryz743r80000gp/T/pip-install-d6fi17h8/scenario_93d92748aa99469aba71337237b49d10/build/lib.macosx-11-x86_64-3.9/scenario -DSCENARIO_CALL_FROM_SETUP_PY:BOOL=ON -DSCENARIO_BUILD_SHARED_LIBRARY:BOOL=OFF -DPython3_EXECUTABLE:PATH=/Users/lorenzorapetti/Software/dic-iit/ADAM/adamEnv/bin/python3.9

==> Building:
$ cmake --build /private/var/folders/v4/8y04pr2912dd1n1pryz743r80000gp/T/pip-install-d6fi17h8/scenario_93d92748aa99469aba71337237b49d10/build/temp.macosx-11-x86_64-3.9_ScenarioCMakeProject --config Release

==> Installing:
$ cmake --install /private/var/folders/v4/8y04pr2912dd1n1pryz743r80000gp/T/pip-install-d6fi17h8/scenario_93d92748aa99469aba71337237b49d10/build/temp.macosx-11-x86_64-3.9_ScenarioCMakeProject

Traceback (most recent call last):
File "/Users/lorenzorapetti/Software/dic-iit/ADAM/adamEnv/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 280, in
main()
File "/Users/lorenzorapetti/Software/dic-iit/ADAM/adamEnv/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 263, in main
json_out['return_val'] = hook(hook_input['kwargs'])
File "/Users/lorenzorapetti/Software/dic-iit/ADAM/adamEnv/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 204, in build_wheel
return _build_backend().build_wheel(wheel_directory, config_settings,
File "/private/var/folders/v4/8y04pr2912dd1n1pryz743r80000gp/T/pip-build-env-r6jc4hrg/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 230, in build_wheel
return self._build_with_temp_dir(['bdist_wheel'], '.whl',
File "/private/var/folders/v4/8y04pr2912dd1n1pryz743r80000gp/T/pip-build-env-r6jc4hrg/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 215, in _build_with_temp_dir
self.run_setup()
File "/private/var/folders/v4/8y04pr2912dd1n1pryz743r80000gp/T/pip-build-env-r6jc4hrg/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 158, in run_setup
exec(compile(code, file, 'exec'), locals())
File "setup.py", line 10, in
setuptools.setup(
File "/private/var/folders/v4/8y04pr2912dd1n1pryz743r80000gp/T/pip-build-env-r6jc4hrg/overlay/lib/python3.9/site-packages/setuptools/init.py", line 159, in setup
return distutils.core.setup(attrs)
File "/usr/local/Cellar/python@3.9/3.9.6/Frameworks/Python.framework/Versions/3.9/lib/python3.9/distutils/core.py", line 148, in setup
dist.run_commands()
File "/usr/local/Cellar/python@3.9/3.9.6/Frameworks/Python.framework/Versions/3.9/lib/python3.9/distutils/dist.py", line 966, in run_commands
self.run_command(cmd)
File "/usr/local/Cellar/python@3.9/3.9.6/Frameworks/Python.framework/Versions/3.9/lib/python3.9/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/private/var/folders/v4/8y04pr2912dd1n1pryz743r80000gp/T/pip-build-env-r6jc4hrg/overlay/lib/python3.9/site-packages/wheel/bdist_wheel.py", line 299, in run
self.run_command('build')
File "/usr/local/Cellar/python@3.9/3.9.6/Frameworks/Python.framework/Versions/3.9/lib/python3.9/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/usr/local/Cellar/python@3.9/3.9.6/Frameworks/Python.framework/Versions/3.9/lib/python3.9/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/usr/local/Cellar/python@3.9/3.9.6/Frameworks/Python.framework/Versions/3.9/lib/python3.9/distutils/command/build.py", line 135, in run
self.run_command(cmd_name)
File "/usr/local/Cellar/python@3.9/3.9.6/Frameworks/Python.framework/Versions/3.9/lib/python3.9/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/usr/local/Cellar/python@3.9/3.9.6/Frameworks/Python.framework/Versions/3.9/lib/python3.9/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/private/var/folders/v4/8y04pr2912dd1n1pryz743r80000gp/T/pip-build-env-r6jc4hrg/overlay/lib/python3.9/site-packages/cmake_build_extension/build_extension.py", line 110, in run
self.build_extension(ext)
File "/private/var/folders/v4/8y04pr2912dd1n1pryz743r80000gp/T/pip-build-env-r6jc4hrg/overlay/lib/python3.9/site-packages/cmake_build_extension/build_extension.py", line 224, in build_extension
subprocess.check_call(configure_command)
File "/usr/local/Cellar/python@3.9/3.9.6/Frameworks/Python.framework/Versions/3.9/lib/python3.9/subprocess.py", line 373, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['cmake', '-S', '/private/var/folders/v4/8y04pr2912dd1n1pryz743r80000gp/T/pip-install-d6fi17h8/scenario_93d92748aa99469aba71337237b49d10', '-B', '/private/var/folders/v4/8y04pr2912dd1n1pryz743r80000gp/T/pip-install-d6fi17h8/scenario_93d92748aa99469aba71337237b49d10/build/temp.macosx-11-x86_64-3.9_ScenarioCMakeProject', '-GNinja', '-DCMAKE_BUILD_TYPE=Release', '-DCMAKE_INSTALL_PREFIX:PATH=/private/var/folders/v4/8y04pr2912dd1n1pryz743r80000gp/T/pip-install-d6fi17h8/scenario_93d92748aa99469aba71337237b49d10/build/lib.macosx-11-x86_64-3.9/scenario', '-DSCENARIO_CALL_FROM_SETUP_PY:BOOL=ON', '-DSCENARIO_BUILD_SHARED_LIBRARY:BOOL=OFF', '-DPython3_EXECUTABLE:PATH=/Users/lorenzorapetti/Software/dic-iit/ADAM/adamEnv/bin/python3.9']' returned non-zero exit status 1.

ERROR: Failed building wheel for scenario
Failed to build scenario
ERROR: Could not build wheels for scenario which use PEP 517 and cannot be installed directly

Environment

OS: MacOS 11.3.1
Python: Python3.9.6

Hi @lrapetti, yes as you read from the support policy, macOS is not currently supported. If you inspect the CMake output, you'll notice that it fails with the following error:

  CMake Error at CMakeLists.txt:205 (message):
    Found no Ignition distribution for PyPI package

You should install Ignition Gazebo manually before attempting to install scenario. Open Robotics currently supports macOS even though I cannot guarantee that scenario will work ootb. If you are willing do iterate on possible errors that might emerge, I can provide support.

I actually already did some work on macOS in #406 while attempting to finalize conda-forge/staged-recipes#16582, but the last update was conda-forge/staged-recipes#16582 (comment) in which everything was building correctly on macOS (a nice start), but tests were segfaulting.

I just remembered that we had a previous discussion regarding macOS in #100, not sure if things have changes since.

Open Robotics currently supports macOS even though I cannot guarantee that scenario will work ootb.

Following the ignition gazebo installation, I was able to install both scenario and gym-ignition via pip without errors.

Still, I am experiencing the command line tool mentioned in #100 (comment). It looks like the problem is still present in the ignition version I have installed.

I was able to run some gym-ignition code such as gym_ignition_models, but not those using gazebo such as the examples:

$python3 launch_cartpole.py
Traceback (most recent call last):
  File "/Users/lorenzorapetti/Software/dic-iit/ADAM/adamEnv/lib/python3.9/site-packages/scenario/__init__.py", line 141, in check_gazebo_installation
    result = subprocess.run(command, capture_output=True, text=True, check=True)
  File "/usr/local/Cellar/python@3.9/3.9.6/Frameworks/Python.framework/Versions/3.9/lib/python3.9/subprocess.py", line 528, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['ign', 'gazebo', '--versions']' returned non-zero exit status 255.

What fails here is just a check for a compatible Ignition Gazebo version, it's not strictly needed during runtime.

Can you please report if the following command executed from a normal terminal succeeds, and if it does, its output?

ign gazebo --versions

If it does not succeed, you can try commenting out the following line in /Users/lorenzorapetti/Software/dic-iit/ADAM/adamEnv/lib/python3.9/site-packages/scenario/__init__.py:

gym-ignition/scenario/bindings/__init__.py

Line 227 in aafd7fb

check_gazebo_installation()

Can you please report if the following command executed from a normal terminal succeeds, and if it does, its output?
ign gazebo --versions

Since there is the problem with the command line tool, I am getting the following

ign gazebo --versions
I cannot find any available 'ign' command:
	* Did you install any ignition library?
	* Did you set the IGN_CONFIG_PATH environment variable?
	    E.g.: export IGN_CONFIG_PATH=$HOME/local/share/ignition

So I commted out the lines as you where suggesting, but I am getting the following

 $python3 launch_cartpole.py
Traceback (most recent call last):
  File "/Users/lorenzorapetti/Software/dic-iit/gym-ignition/examples/python/launch_cartpole.py", line 9, in <module>
    from gym_ignition.utils import logger
  File "/Users/lorenzorapetti/Software/dic-iit/ADAM/adamEnv/lib/python3.9/site-packages/gym_ignition/__init__.py", line 14, in <module>
    from gym_ignition.utils import resource_finder
  File "/Users/lorenzorapetti/Software/dic-iit/ADAM/adamEnv/lib/python3.9/site-packages/gym_ignition/utils/__init__.py", line 5, in <module>
    from . import logger, math, misc, resource_finder, scenario
  File "/Users/lorenzorapetti/Software/dic-iit/ADAM/adamEnv/lib/python3.9/site-packages/gym_ignition/utils/math.py", line 10, in <module>
    from scenario import gazebo as scenario
  File "/Users/lorenzorapetti/Software/dic-iit/ADAM/adamEnv/lib/python3.9/site-packages/scenario/__init__.py", line 228, in <module>
    import_gazebo()
  File "/Users/lorenzorapetti/Software/dic-iit/ADAM/adamEnv/lib/python3.9/site-packages/scenario/__init__.py", line 203, in import_gazebo
    import scenario.bindings.gazebo
  File "/Users/lorenzorapetti/Software/dic-iit/ADAM/adamEnv/lib/python3.9/site-packages/scenario/bindings/gazebo.py", line 13, in <module>
    from . import _gazebo
ImportError: dlopen(/Users/lorenzorapetti/Software/dic-iit/ADAM/adamEnv/lib/python3.9/site-packages/scenario/bindings/_gazebo.so, 10): Library not loaded: /usr/local/opt/urdfdom/lib/liburdfdom_sensor.3.0.dylib
  Referenced from: /usr/local/opt/sdformat12/lib/libsdformat12.12.dylib
  Reason: image not found

I guess that is due to some conflict or wrong configuration on my laptop. Indeed I see the following

$ ls /usr/local/opt/urdfdom/lib
liburdfdom_model.1.0.dylib		liburdfdom_sensor.dylib
liburdfdom_model.dylib			liburdfdom_world.1.0.dylib
liburdfdom_model_state.1.0.dylib	liburdfdom_world.dylib
liburdfdom_model_state.dylib		pkgconfig
liburdfdom_sensor.1.0.dylib		urdfdom

I see that you have liburdfdom_sensor.1.0.dylib while sdformat12 requires liburdfdom_sensor.3.0.dylib. Could it be a problem of the packages you already had installed in your system before installing Ignition Gazebo? Can you try to update the library?

Another option is to try using a conda environment instead of relying on the official macOS packages. Though, even with conda, something could be missing.

Not sure if this is the right place for reporting experimental environments, but I have had success in getting ScenarIO running on macOS (with GUI) using development branches that add Metal support to Ignition (which will be slowly making there way into released versions - hopefully in time for Ignition Garden). I'd be happy to outline the approach here or somewhere suitable if there is an interest. This looks like a great tool btw! I expect to be using it a good deal.

Hi @srmainwaring, this is great news, thanks! I'm not sure if @lrapetti has any news after his last attempts, in any case it would be very helpful if you can describe your working setup and whether there's anything missing on our side to run on macOS.

I'm not sure if @lrapetti has any news after his last attempts

Sorry, I didn't proceed further at this stage. In case I'll let you know when I start again this investigation.

Hi @diegoferigo - here's the description, it may take a number of edits to get right as it is fairly involved at this stage. I hope it will simplify as more of the development work is merged into the relevant repos.

Install Ignition on macOS

The first step is to get the Ignition graphics API running properly on macOS. I've documented how to install Ignition with Metal rendering on macOS here: https://github.com/srmainwaring/ign-rendering/wiki. Over time this page should become redundant as more features are merged into the https://github.com/ignitionrobotics repos.

Install iDynTree

The installation on macOS largely follows the iDynTree install instructions with an extra step to install to /usr/local/Cellar/idyntree/1.0.0 and use brew to link into /usr/local.

Clone and install:

# clone to ~/robotology
$ cd ~/robotology
$ git clone https://github.com/robotology/idyntree.git
$ cd idyntree
$ mkdir build && cd build
$ cmake .. \
  -DCMAKE_INSTALL_PREFIX=/usr/local/Cellar/idyntree/1.0.0 \
  -DCMAKE_BUILD_TYPE=RelWithDebInfo \
  -DIDYNTREE_USES_PYTHON=ON
$ make -j16
$ make install
$ brew link idyntree

Check the installation:

>>> import idyntree
>>> import idyntree.bindings as iDynTree
>>> p = iDynTree.Position()
>>> print(p)
1.46135e+224 1.0947e+248 1.02265e-259

Run the example:

$ cd ~/robotology/idyntree/examples/python
$ python KinDynComputationsTutorial.py

Install ScenarIO

We use a patched version of ScenarIO which includes dependency definitions for Ignition Garden. The libraries are installed to /usr/local/Cellar/scenario/1.0.0 and then brew linked into /usr/local:

# clone to ~/robotology
$ cd ~/robotology
$ git clone https://github.com/robotology/gym-ignition.git
$ git remote add fork https://github.com/srmainwaring/gym-ignition.git
$ git checkout fork/feature/ign-garden-macos
$ cd gym-ignition/scenario
$ mkdir build && cd build
$ cmake .. \
  -DCMAKE_INSTALL_PREFIX=/usr/local/Cellar/scenario/1.0.0 \
  -DCMAKE_BUILD_TYPE=RelWithDebInfo \
  -DSCENARIO_USE_IGNITION=ON \
  -DSCENARIO_ENABLE_BINDINGS=ON
$ make -j16
$ make install
$ brew link scenario
$ cd ~/robotology/gym-ignition
$ pip install --user -e scenario
$ pip install --user -e .

I have not used a Python virtual environment for my installation, but that should also work.

Configuration

ScenarIO uses a custom physics plugin that overrides the defaults in World.cpp

gym-ignition/scenario/src/gazebo/src/World.cpp

Lines 251 to 284 in fa14f7f

    
           bool World::setPhysicsEngine(const PhysicsEngine engine) 
        
           { 
        
               // Get the name of the physics plugin 
        
               const std::string pluginLib = [&engine]() -> std::string { 
        
                   switch (engine) { 
        
                       case PhysicsEngine::Dart: 
        
                           return "ignition-physics" 
        
                                  + std::to_string(IGNITION_PHYSICS_MAJOR_VERSION) 
        
                                  + "-dartsim-plugin"; 
        
                   } 
        
                   return ""; 
        
               }(); 
        
               if (pluginLib.empty()) { 
        
                   sError << "Failed to retrieve the name of physics plugin library"; 
        
                   return false; 
        
               } 
        
               // This component is read by the Physics system during its configuration 
        
               utils::setComponentData<ignition::gazebo::components::PhysicsEnginePlugin>( 
        
                   m_ecm, m_entity, pluginLib); 
        
               // Vendored Physics system 
        
               const std::string libName = "PhysicsSystem"; 
        
               const std::string className = "scenario::plugins::gazebo::Physics"; 
        
               // Load the Physics system 
        
               if (!this->insertWorldPlugin(libName, className)) { 
        
                   sError << "Failed to insert the physics plugin" << std::endl; 
        
                   return false; 
        
               } 
        
               return true; 
        
           }

It expects to find the system plugin libPhysicsSystem.dylib so we need to add the install location to IGN_GAZEBO_PLUGIN_INSTALL_DIR when running C++ scenarios:

export IGN_GAZEBO_PLUGIN_INSTALL_DIR=${IGN_GAZEBO_PLUGIN_INSTALL_DIR}:/usr/local/lib/scenario/plugins

Testing

Run the pendulum Python example from VSCode. Set up a launch configuration entry in ~/robotology/.vscode/launch.json:

{
  "version": "0.2.0",
  "configurations": [
    {
      "name": "Python: ScenarIO",
      "type": "python",
      "request": "launch",
      "program": "${file}",
      "console": "integratedTerminal",
      "env": {
        "IGN_VERBOSE": "0",
        "IGN_CONFIG_PATH": "/usr/local/share/ignition",
        "IGN_RENDERING_RESOURCE_PATH": "/usr/local/share/ignition/ignition-rendering7",
      },
    },
  ]
}

Add the following lines to the pendulum example to ensure that ignition uses the brew installed version of ruby:

# ensure we are using /usr/local/opt/ruby/bin/ruby instead of the system version
import os
PATH = os.getenv("PATH")
PATH = "/usr/local/opt/ruby/bin:" + PATH 
os.environ["PATH"] = PATH

Then run the example:

Wow @srmainwaring great work, nicely done! Thanks for this big effort to smooth the macOS experience. I don't have a macOS machine to test everything myself, but I will eagerly wait for your contributions to land upstream and then giving another shot in the conda-forge packaging I drafted in conda-forge/staged-recipes#16582 (unfortunately I'm a bit busy lately and I have to postpone it a little longer). As you noted, from the ScenarIO point of view, changes are minimal, only the introduction of Garden support was missing. I can surely add it to the nightly channel so that your setup could get rid of a fork, and those that are interested in building the project from source can benefit from it! And, actually, note to myself, it would a good idea in the future adding a similar CMake support of the development Ignition distribution right after the previous upstream stable version is released.

	bool World::setPhysicsEngine(const PhysicsEngine engine)
	{
	// Get the name of the physics plugin
	const std::string pluginLib = [&engine]() -> std::string {
	switch (engine) {
	case PhysicsEngine::Dart:
	return "ignition-physics"
	+ std::to_string(IGNITION_PHYSICS_MAJOR_VERSION)
	+ "-dartsim-plugin";
	}
	return "";
	}();

	if (pluginLib.empty()) {
	sError << "Failed to retrieve the name of physics plugin library";
	return false;
	}

	// This component is read by the Physics system during its configuration
	utils::setComponentData<ignition::gazebo::components::PhysicsEnginePlugin>(
	m_ecm, m_entity, pluginLib);

	// Vendored Physics system
	const std::string libName = "PhysicsSystem";
	const std::string className = "scenario::plugins::gazebo::Physics";

	// Load the Physics system
	if (!this->insertWorldPlugin(libName, className)) {
	sError << "Failed to insert the physics plugin" << std::endl;
	return false;
	}

	return true;
	}