Install without conda

Question

Install without conda

shuocao818 opened this issue 5 years ago · 11 comments

Hi, is it possible to install pyembree without using conda?
I tried python setup.py install and also I set the include path of my embree in the setup.py like this
include_path = [np.get_include(),"C:\python36\Lib\site-packages\embree-2.14.0.x64.windows\include"]
but I got an error: fatal error LNK1181: cannot open input file 'embree.lib'
so I am wondering whether conda is the only way to install it. Thanks!

Answer 1 · 2020-05-25T02:45:52.000Z

Yeah, you should be able to install it without conda, but you'll need to install embree seprately and then point pip/setup.py install to the location you installed embree at

Answer 2 · 2020-06-05T13:23:56.000Z

@shuocao818 go into the pyembree folder and run the following command and it should work without issues:
pip install --global-option=build_ext --global-option="-IC:\Program Files\Intel\Embree v2.17.7 x64\include" --global-option="-LC:\Program Files\Intel\Embree v2.17.7 x64\lib" .

just modify your embree installation folder path.

Answer 3 · 2020-06-14T04:35:14.000Z

Thank you so much!

Answer 4 · 2020-06-22T18:53:40.000Z

For whatever it's worth we install pyembree/embree 2.17.7 on the trimesh docker images (vanilla Python + Debian buster) using this bash script:

wget https://github.com/mikedh/trimesh/blob/master/docker/builds/embree.bash
sudo bash embree.bash

Answer 5 · 2020-06-28T16:44:37.000Z

@shuocao818 go into the pyembree folder and run the following command and it should work without issues:
pip install --global-option=build_ext --global-option="-IC:\Program Files\Intel\Embree v2.17.7 x64\include" --global-option="-LC:\Program Files\Intel\Embree v2.17.7 x64\lib" .

just modify your embree installation folder path.

After changing to embree installation path, I get the following error.

pyembree/mesh_construction.cpp:644:10: fatal error: 'embree2/rtcore.h' file not found #include "embree2/rtcore.h" ^~~~~~~~~~~~~~~~~~ 1 warning and 1 error generated. error: command 'clang' failed with exit status 1

-----edit-------
Issue is the path embree2/* doesn't exist with version upgrade and download of embree3.
To keep up with each release of embree, you will have to change those references.

Answer 6 · 2020-06-30T08:05:23.000Z

@yehDilBeparwah but since you were linking to the embree v2.17.7 the folder embree2 should be there.

Answer 7 · 2021-02-02T02:08:00.000Z

I installed it by going to https://github.com/mikedh/trimesh/blob/master/docker/builds/embree.bash, copying the script into a file named embree.bash, and running sudo bash embree.bash. Then, I added
export LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH
to my .bashrc and I can use pyembree

Answer 8 · 2021-07-19T18:48:47.000Z

@ttsesm @Shivin302 @yeDilBeparwah @mikedh I am trying to install pyembree without conda as well. So far, my steps have been:

System: Windows 10 x64, Build 1909, Python 3.8.10

Install Embree v3.13.0 via the msi installer
Install Microsoft C++ Build Tools for Visual Studio Community 2019, including
(a) Desktop development with C++
(b) .NET desktop build tools
(c) Universal Windows Platform build tools
(d) .NET build tools
(e) Vistual Studio extension development
Install the Windows 10.0.19041.685 SDK
Add the following to the system path:

C:\Program Files (x86)\Microsoft Visual Studio\2019\Community
C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools
C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.29.30037\bin\Hostx64\x64
C:\Program Files (x86)\Windows Kits\10
C:\Program Files (x86)\Windows Kits\10\Lib\10.0.10240.0\ucrt\x64\
C:\Program Files (x86)\Windows Kits\10\Include\10.0.19041.0\

Install poetry via PowerShell
Create a poetry 1.1.7 project in a folder (%USERPROFILE%\Code\myproject) with a venv virtual environment in subfolder .venv
Clone the pyembree repo to %USERPROFILE%\Code\myproject\.venv\Lib\site-packages\ (creates a subfolder pyembree): git clone https://github.com/scopatz/pyembree.git
Open a Developer Command Prompt for VS 2019, which activates vcvarsall.bat for x64 build
Run the following:

set INCLUDE="C:\Program Files\Intel\Embree3\include\embree3\"
set LIB="C:\Program Files\Intel\Embree3\lib\"
cd "%USERPROFILE%\Code\myproject"
.venv\Scripts\activate.bat
cd ".venv\Lib\site-packages\pyembree"
py setup.py install

However, no matter what, I get the following output:

Warning: passing language='c++' to cythonize() is deprecated. Instead, put "# distutils: language=c++" in your .pyx or .pxd file(s)
running install
running bdist_egg
running egg_info
writing pyembree.egg-info\PKG-INFO
writing dependency_links to pyembree.egg-info\dependency_links.txt
writing top-level names to pyembree.egg-info\top_level.txt
reading manifest file 'pyembree.egg-info\SOURCES.txt'
adding license file 'LICENSE'
adding license file 'AUTHORS'
writing manifest file 'pyembree.egg-info\SOURCES.txt'
installing library code to build\bdist.win-amd64\egg
running install_lib
running build_py
running build_ext
building 'pyembree.mesh_construction' extension
error: Microsoft Visual C++ 14.0 or greater is required. Get it with "Microsoft C++ Build Tools": https://visualstudio.microsoft.com/visual-cpp-build-tools/

Any ideas what I'm doing wrong?

Answer 9 · 2021-07-19T19:03:12.000Z

Is part of my issue that I'm using Embree 3 and not Embree 2?

Answer 10 · 2021-07-20T01:30:35.000Z

Installation Instructions

Tested on:

OS: Windows 10 x64 Professional, Build 1909
Python: 3.8.5 x64
Date: 31-JULY-2021

Steps

Install Microsoft Visual Studio 2017 Build Tools and the Windows 10 SDK. These are required for building cython code.

(NOTE: FYI, Microsoft Visual Studio version numbers are not the same as Microsoft Visual C++ Compiler versions. MSVCv14X x64/x86 build tools are required for Python 3.8. Installing the Visual Studio 2017 Build Tools with the Windows 10 SDK. If you wish to install Visual Studio 2017, select the Desktop development with C++ Workload and make sure the required Windows 10 SDK build tool is selected under Individual Components. Refer to the Python Wiki for the right build tool versions to install).

Install vcpkg in C:\vcpkg and add the path to your System Environment Variables. Be sure to bootstrap and integrate vcpkg:

cd C:\vcpkg
git clone https://github.com/Microsoft/vcpkg.git
bootstrap-vcpkg.bat
vcpkg integrate install

Alternatively, you can download and install the embree-2.17.7.x64.msi Installer on Embree's GitHub page (to learn more about Embree, visit their website).

Install embree2 64-bit. If using vcpkg, this is done from the command-line as:

vcpkg install embree2:x64-windows

NOTE: To date, pyembree relies on Embree 2 and has not been updated version 3. The above command will install version 2.17.7, which is the required version.

Create your project folder and initialize a virtual environment with venv. In this example, Python 3.8.5 x64-bit is chosen, simply as it is the Python version used in Miniconda py38_4.9.2. (It may work for Python 3.6 and 3.7, but at the time of this writing, one should not go to higher than 3.8)
Install the following packages:

py -m pip install numpy cython wheel setuptools pyvista pykdtree rtree trimesh

Navigate to <virtual environment folder>\Lib\site-packages and clone the pyembree repo:

git clone https://github.com/scopatz/pyembree.git

and remove the underlying .git subfolder.

Move into pyembree folder and modify the setup.py file to:

import os
from setuptools import find_packages, setup

import numpy as np
from Cython.Build import cythonize
from Cython.Distutils import build_ext


include_path = [
    np.get_include(),
]

ext_modules = cythonize("pyembree/*.pyx", language_level=3, include_path=include_path)

for ext in ext_modules:
    ext.include_dirs = include_path
    ext.libraries = [
        "pyembree/embree2/lib/embree",
        "pyembree/embree2/lib/tbb",
        "pyembree/embree2/lib/tbbmalloc",
    ]

setup(
    name="pyembree",
    version="0.1.6",
    cmdclass={"build_ext": build_ext},
    ext_modules=ext_modules,
    zip_safe=False,
    packages=find_packages(),
    include_package_data=True,
    package_data={"pyembree": ["*.cpp", "*.dll"]},
)

Move into the pyembree subfolder. Copy the folder C:\vcpkg\installed\x64-windows\include\embree2 into this folder (if you installed from the .msi, this will be the folder C:\Program Files\Intel\Embree v2.17.7 x64\include\embree2). In addition, copy the libraries from C:\vcpkg\installed\x64-windows\lib into this embree2 folder (if you installed from the .msi, this will be the folder C:\Program Files\Intel\Embree v2.17.7 x64\lib\). You only need the (static) libraries

embree.lib
tbb.lib
tbbmalloc.lib

and dynamic libraries

embree.dll
tbb.dll
tbbmalloc.dll

Add the following line to the top of every *.pyx and *.pxd file in pyembree:

# distutils: language=c++

and change every relative import of an rtcore module to an absolute import in each file. For example, change:

cimport rtcore as rtc

to

cimport pyembree.rtcore as rtc

Create the following MANIFEST.in file in the top-level pyembree folder:

include *.h *.pxd *.dll
recursive-include embree2 *.h *.isph
recursive-include embree2/lib *.lib

At the end, your folder should look like this:

<virtual environment folder>\Lib\site-packages\pyembree
│   .authors.yml
│   .gitignore
│   .mailmap
│   .travis-install.sh
│   .travis.yml
│   AUTHORS
│   CHANGELOG.rst
│   LICENSE
│   MANIFEST.in
│   README.rst
│   rever.xsh
│   setup.py
│   
├───examples
│       attenuate.py
│       intersection.py
│       
├───news
│       TEMPLATE.rst
│       
├───pyembree
│   │   embree.dll
│   │   mesh_construction.h
│   │   mesh_construction.pyx
│   │   rtcore.pxd
│   │   rtcore.pyx
│   │   rtcore_geometry.pxd
│   │   rtcore_geometry_user.pxd
│   │   rtcore_ray.pxd
│   │   rtcore_scene.pxd
│   │   rtcore_scene.pyx
│   │   tbb.dll
│   │   tbbmalloc.dll
│   │   triangles.pyx
│   │   __init__.pxd
│   │   __init__.py
│   │
│   └───embree2
│       │   rtcore.h
│       │   rtcore.isph
│       │   rtcore_builder.h
│       │   rtcore_geometry.h
│       │   rtcore_geometry.isph
│       │   rtcore_geometry_user.h
│       │   rtcore_geometry_user.isph
│       │   rtcore_ray.h
│       │   rtcore_ray.isph
│       │   rtcore_scene.h
│       │   rtcore_scene.isph
│       │   rtcore_version.h
│       │
│       └───lib
│               embree.lib
│               tbb.lib
│               tbbmalloc.lib
│
├───recipes
│   └───pyembree
│           build.sh
│           meta.yaml
│
└───tests
        test_intersection.py

Build pyembree by running the following from the top-level pyembree folder (TIP: To help debug errors, pipe stderr to a local file errors.txt that you can look at):

py setup.py build_ext -i 2> errors.txt

Test that everything is working correctly by opening a python terminal in your virtual environment and running

>>> import pyembree
>>> from pyembree import rtcore_scene
>>>

If you get no errors, then pyembree has been installed correctly. You can also create a .egg and install the library with

py setup.py install

The setup.py we created will copy the dlls and cpp files to the correct folder.

As an last optional step, create a main.py in your virtual environment and copy and paste the code from Project to Finite Plane. If it runs without errors in 1-2 seconds, then pyembree is working properly with trimesh for vectorized ray tracing (NOTE: trimesh will work without pyembree, but it will be very slow. It should not take a long time to run this example code if pyembree is properly installed. trimesh resorts back to trimesh.ray.ray_triangle.RayMeshIntersector if it cannot import pyembree (see pyembree ray casting acceleration #875), so you won't see any errors; just slow running code):

import numpy as np
from pykdtree.kdtree import KDTree

import pyvista as pv
from pyvista import examples

# Load data
data = examples.load_random_hills()
data.translate((10, 10, 10))

# Create triangular plane (vertices [10, 0, 0], [0, 10, 0], [0, 0, 10])
size = 10
vertices = np.array([[size, 0, 0], [0, size, 0], [0, 0, size]])
face = np.array([3, 0, 1, 2])

planes = pv.PolyData(vertices, face)

# Subdivide plane so we have multiple points to project to
planes = planes.subdivide(8)

# Get origins and normals
origins = planes.cell_centers().points
normals = planes.compute_normals(cell_normals=True, point_normals=False)["Normals"]

# Vectorized Ray trace
points, pt_inds, cell_inds = data.multi_ray_trace(
    origins, normals
)  # Must have rtree, trimesh, and pyembree installed

# Filter based on distance threshold, if desired (mimics VTK ray_trace behavior)
# threshold = 10  # Some threshold distance
# distances = np.linalg.norm(origins[inds] - points, ord=2, axis=1)
# inds = inds[distances <= threshold]

tree = KDTree(data.points.astype(np.double))
_, data_inds = tree.query(points)

elevations = data.point_arrays["Elevation"][data_inds]

# Mask points on planes
planes.cell_arrays["Elevation"] = np.zeros((planes.n_cells,))
planes.cell_arrays["Elevation"][pt_inds] = elevations
planes.set_active_scalars("Elevation")  # Probably not necessary, but just in case

# Create axes
axis_length = 20
tip_length = 0.25 / axis_length * 3
tip_radius = 0.1 / axis_length * 3
shaft_radius = 0.05 / axis_length * 3
x_axis = pv.Arrow(
    direction=(axis_length, 0, 0),
    tip_length=tip_length,
    tip_radius=tip_radius,
    shaft_radius=shaft_radius,
    scale="auto",
)
y_axis = pv.Arrow(
    direction=(0, axis_length, 0),
    tip_length=tip_length,
    tip_radius=tip_radius,
    shaft_radius=shaft_radius,
    scale="auto",
)
z_axis = pv.Arrow(
    direction=(0, 0, axis_length),
    tip_length=tip_length,
    tip_radius=tip_radius,
    shaft_radius=shaft_radius,
    scale="auto",
)
x_label = pv.PolyData([axis_length, 0, 0])
y_label = pv.PolyData([0, axis_length, 0])
z_label = pv.PolyData([0, 0, axis_length])
x_label.point_arrays["label"] = [
    "x",
]
y_label.point_arrays["label"] = [
    "y",
]
z_label.point_arrays["label"] = [
    "z",
]

# Plot results
p = pv.Plotter()
p.add_mesh(x_axis, color="r")
p.add_point_labels(x_label, "label", show_points=False, font_size=24)
p.add_mesh(y_axis, color="r")
p.add_point_labels(y_label, "label", show_points=False, font_size=24)
p.add_mesh(z_axis, color="r")
p.add_point_labels(z_label, "label", show_points=False, font_size=24)
p.add_mesh(data)
p.add_mesh(planes)
p.show()

Answer 11 · 2023-09-08T19:47:36.000Z

After a while of struggling with installing pyembree on Windows, I found this instruction more straightforward and included only the necessary steps to install the package.

TESTED ON WINDOWS 10 SEP 2023

go to the embree releases page , Download and install using the .msi version.
Use your desired conda environment or create and activate a new one.
Install Microsoft visual studio (version >= 14) and choose the "Desktop development with C++" workload while installing.
Install the following packages:

cython==0.29.35 \\ newer versions don't work
numpy
setuptools

Clone pyembree to the site-packages directory of your environment. Here is the default address in Windows.
cd C:\Users\User\anaconda3\envs\<env_name>\Lib\site-packages
then
git clone https://github.com/scopatz/pyembree.git
copy "embree2" folder from <embree_installation_path>\include to the C:\Users\User\anaconda3\envs\<env_name>\Lib\site-packages\pyembree\pyembree directory.
Hint: Default <embree_installation_path> is C:\Program Files\Intel\Embree v2.17.6 x64
Copy everything located in <embree_installation_path> to the <anaconda_installation_path>\envs\<env_name>\libs
Find the following files in the <embree_installation_path>\bin\ and copy them to <anaconda_installation_path>\envs\<env_name>\Library\bin\.

 - embree.dll
 - tbb.dll
 - tbbmalloc.dll

Make sure your pyembree directory looks similar the the following folder structure:

Pyembree
|----examples
|----news
|----receipts
|----tests
|----pyembree
|      |____ embree2
|             |---- rtcore_builder.h
|             |---- rtcore.h
|             |---- rtcores..isph
|             |---- rtcore_geometry.h
|             |---- rtcore_geometry.isph
|             |---- rtcore_geometry_user.h
|             |---- rtcore_ray.h
|             |---- rtcore_ray.isph
|             |---- rtcore_scene.h
|             |---- rtcore_scene.isph
|             |---- rtcore_version.h
|----.authors.yml
.
.
.

Make sure your conda environment is active and go to the pyembree directory and run:
pip install .
Test your installation by running the following codes on the python interpreter:

import pyembree
from pyembree import rtcore

If you face an error in running the second line, try restarting your environment/system.
Note: Just use pyembree for validation and experiments. don't use it in deployment and try to find more stable alternatives.