RenderKit/embree

Using embree4 with open3d produces a random raycasting result

Closed this issue · 6 comments

elias-Mimetrik commented

embree version: 4.3.3
open3d version: 0.18

This is a simple embree example, I create a sphere in open3d, and launch a ray towards it using embree4.

The result is random: once I get an intersection, once I get an infinity value:

If I switch to embree3, all works well.

#include <iostream>
#include <limits>
#include <vector>

#include <embree4/rtcore.h>

#include <open3d/Open3D.h>

int main()
{
    std::shared_ptr<open3d::geometry::TriangleMesh> mesh = 
        open3d::geometry::TriangleMesh::CreateSphere(10.0);

    RTCDevice device = rtcNewDevice(NULL);
    RTCScene scene_embree = rtcNewScene(device);
    RTCGeometry geometry_top = rtcNewGeometry(device, RTC_GEOMETRY_TYPE_TRIANGLE);

    float* buffer_vertices_top = (float*) rtcSetNewGeometryBuffer(geometry_top, 
                                                                  RTC_BUFFER_TYPE_VERTEX, 
                                                                  0, RTC_FORMAT_FLOAT3,
                                                                  3*sizeof(float),
                                                                  mesh->vertices_.size());

    for (unsigned int i = 0; i < mesh->vertices_.size(); ++i)
    {
        Eigen::Vector3f pt = mesh->vertices_[i].cast<float>();
        buffer_vertices_top[(i * 3)]     = pt.x();
        buffer_vertices_top[(i * 3) + 1] = pt.y();
        buffer_vertices_top[(i * 3) + 2] = pt.z();
    }

    unsigned* buffer_indices_top = (unsigned*) rtcSetNewGeometryBuffer(geometry_top, 
                                                     RTC_BUFFER_TYPE_INDEX, 0, RTC_FORMAT_UINT3,
                                                     3*sizeof(unsigned), mesh->triangles_.size());

    for (unsigned int i = 0; i < mesh->triangles_.size(); ++i)
    {
        Eigen::Vector3i ids = mesh->triangles_[i];
        buffer_indices_top[(i * 3)]     = ids.x();
        buffer_indices_top[(i * 3) + 1] = ids.y();
        buffer_indices_top[(i * 3) + 2] = ids.z();
    }

    rtcCommitGeometry(geometry_top);
    rtcAttachGeometry(scene_embree, geometry_top);
    rtcReleaseGeometry(geometry_top);
    rtcCommitScene(scene_embree);

    float* ptr = static_cast<float*>(rtcGetGeometryBufferData(geometry_top, 
                                                    RTC_BUFFER_TYPE_VERTEX,0));
    unsigned* ptr2 = static_cast<unsigned*>(rtcGetGeometryBufferData(geometry_top, 
                                                    RTC_BUFFER_TYPE_INDEX,0));

    Eigen::Vector3f camera_centre(0, 0, 35);
    Eigen::Vector3f current_vertex(2,0,11);

    RTCRayHit rayhit;
    rayhit.ray.org_x = camera_centre.x();
    rayhit.ray.org_y = camera_centre.y();
    rayhit.ray.org_z = camera_centre.z();

    Eigen::Vector3f dir_cam_to_vertex = current_vertex - camera_centre;
    dir_cam_to_vertex.normalize();
    rayhit.ray.dir_x = dir_cam_to_vertex.x();
    rayhit.ray.dir_y = dir_cam_to_vertex.y();
    rayhit.ray.dir_z = dir_cam_to_vertex.z();

    rayhit.ray.tnear = 0.f;
    rayhit.ray.tfar = std::numeric_limits<float>::infinity();
    rayhit.hit.geomID = RTC_INVALID_GEOMETRY_ID;

    RTCRayQueryContext context;
    rtcInitRayQueryContext(&context);
    RTCIntersectArguments intersectArgs;
    rtcInitIntersectArguments(&intersectArgs);
    intersectArgs.context = &context;
    rtcIntersect1(scene_embree, &rayhit, &intersectArgs);

    if (rayhit.hit.geomID != RTC_INVALID_GEOMETRY_ID)
    {
        std::cout << "Intersection at t (distance) = " << rayhit.ray.tfar << std::endl;
    }
    else
    {
        std::cout << "No Intersection =" << rayhit.ray.tfar << "\n\n";
    }

    rtcReleaseScene(scene_embree);
    rtcReleaseDevice(device);
}
freibold commented

Hi,

can you tell me a bit more about the OS and compiler you're using. I've tried to reproduce on my Windows machine with VS2022 but I see strange things already with
std::shared_ptr<open3d::geometry::TriangleMesh> mesh = open3d::geometry::TriangleMesh::CreateSphere(10.0);
When I build in Debug mode your reproducer will segfault on my machine and if I print mesh->vertices_.size() after the above line it says 18446743969357181358 which indicates some problem with uninitialized memory to me which in turn could explain why you see non-deterministic behaviour. In Release mode your example runs through and always prints No Intersection =inf on my machine.

freibold commented

And I just tried the same with a Embree 3 version. I see the same behaviour with mesh->vertices_.size() returning some garbage value (1844674xxxxxxxxxxxxx).

elias-Mimetrik commented

Thanks for your reply. I am on ubuntu 22.04.

Creation of the sphere mesh is the standard way of creating a mesh (see the docs) it should not cause any problems. It does not give me any problems. Perhaps you need an extra include on windows #include "open3d/geometry/TriangleMesh.h"

The ray is pointed directly at the sphere (you can print and check the values). But despite this I get random behaviour. But everything works fine when I switch from embree v4.3.3 to v3.13.5. In both versions I'm using the same open3d v0.18.0 which you used.

I'm doubting some interference between open3d and embree since open3d uses embree internally. Thanks.

Update:
If I print mesh->vertices_.size() I always get 762.

freibold commented

I think I've found the problem. You have to set rayhit.ray.mask to -1 like in the minimal tutorial here:

embree/tutorials/minimal/minimal.cpp

Line 171 in 5730b15

rayhit.ray.mask = -1;
. With that change I reliably get the intersection at t = 25.3403. Let me know if that work for you.

You definitely have an issue on Windows in Debug mode though.

  • 👍1
elias-Mimetrik commented

I confirm that this is indeed the issue. My complete example runs fine now. Thanks very much.