A unified library for 3D data processing and analysis with both C++&Python API based on Open3D.
This library aims at providing some useful 3d processing algorithms which Open3D is not yet provided or not easy to use, and sharing the same data structures used in Open3D.
Core modules:
-
common
:- Normals estimaiton from PointMap
- Ransac for primitives fitting, including plane, sphere and cylinder, with parallel computing supported.
- K nearest neighbors search based on annoy. It has the similar API as
open3d.geometry.KDTreeFlann
class (the radius search is not supported).
-
preprocessing
:- Farthest point sampling
- Crop ROI of point clouds.
- Project point clouds into a plane.
-
features
:- Boundary points detection from point clouds.
-
registration
:- Corresponding matching with descriptors.
- 3D rigid transformation solver including SVD, RANSAC and TEASERPP.
-
pose_estimation
:- Point Pair Features (PPF) based 6D pose estimator. (This implementation is evaluated on Linemod, Linemod-Occluded and YCB-Video dataset, the performance can be found in BOP Leaderboards/PPF-3D-ICP)
- A RayCastRenderer, which is useful for partial view point clouds, depth map and instance map generation.
-
segmentation
:- Proximity extraction in scalable implementation with different vriants, including distance, and normal angle.
- Plane segementation using iterative ransac plane fitting.
-
reconstruction
- RGBD dense reconstruction, an improved version of Open3D legacy reconstruction system. See toturial for more details.
cmake
>= 3.10python
>= 3.6eigen
>= 3.3open3d
>= 0.14.1pybind11
>= 2.6.2
- Build
open3d
as external library. You can follow the instruction from here guide. Buildpybind11
in your system as well. If you only use C++ API, you can skip this step and just download the pre-builtopen3d
library from official website.
-
Git clone the repo and run:
mkdir build && cd build cmake .. -DCMAKE_INSTALL_PREFIX=</path/to/installation> make install -j
If you only use C++ API, make sure you add
-DBUILD_PYTHON=OFF
. -
After installation, add these two lines to
~/.bashrc
file:# this is not necessary if you do not build python binding export PYTHONPATH="$PYTHONPATH:</path/to/installation>/misc3d/lib/python" # this is necessary for c++ to find the customized installation library export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:</path/to/installation>/misc3d/lib"
Run
source ~/.bashrc
to save changes.
Note: In windows, you can download the pre-build Open3D c++ Release and install python package via pip3 install open3d
.
-
(optional)) Git clone and run:
mkdir build && cd build
. You can use Cmake GUI to configure your build options. Then runcmake --build . --config Release --target INSTALL
to installMisc3D
. -
After installation, add this variable:
/path/to/installation/misc3d/lib/python
to your system environment variablePath
to make sure you can importmisc3d
in python.
The example python scripts can be found in examples/python
. You can run it after you install the library successfully.
You can import misc3d
same as open3d
:
import open3d as o3d
import misc3d as m3d
# Estimate normals inplace.
m3d.common.estimate_normals(pcd, (640, 480), 3)
# Ransac for primitives fitting.
w, indices = m3d.common.fit_plane(pcd, 0.01, 100)
w, indices = m3d.common.fit_sphere(pcd, 0.01, 100)
w, indices = m3d.common.fit_cylinder(pcd, 0.01, 100)
# Farthest point sampling.
indices = m3d.preprocessing.farthest_point_sampling(pcd, 1000)
# Crop ROI point clouds.
pcd_roi = m3d.preprocessing.crop_roi_pointcloud(pcd, (500, 300, 600, 400), (640, 480))
# Project point clouds into a plane.
pcd_plane = m3d.preprocessing.project_into_plane(pcd)
# Boundary points detection.
index = m3d.features.detect_boundary_points(
pcd, o3d.geometry.KDTreeSearchParamHybrid(0.02, 30))
boundary = pcd.select_by_index(index)
# Features matching using FLANN or ANNOY
# `fpfh_src` is open3d.pipeline.registration.Feature instance which is computed using FPFH 3d descriptor.
index1, index2 = m3d.registration.match_correspondence(fpfh_src, fpfh_dst, m3d.registration.MatchMethod.FLANN)
index1, index2 = m3d.registration.match_correspondence(fpfh_src, fpfh_dst, m3d.registration.MatchMethod.ANNOY)
# Solve 3d rigid transformation.
# Ransac solver
pose = m3d.registration.compute_transformation_ransac(pc_src, pc_dst, (index1, index2), 0.03, 100000)
# SVD solver
pose = m3d.registration.compute_transformation_svd(pc_src, pc_dst)
# Teaser solver
pose = m3d.registration.compute_transformation_teaser(pc_src, pc_dst, 0.01)
# PPF pose estimator.
# Init config for ppf pose estimator.
config = m3d.pose_estimation.PPFEstimatorConfig()
config.training_param.rel_sample_dist = 0.04
config.score_thresh = 0.1
config.refine_param.method = m3d.pose_estimation.PPFEstimatorConfig.PointToPlane
ppf = m3d.pose_estimation.PPFEstimator(config)
ret = ppf.train(model)
ret, results = ppf.estimate(scene)
# Create a ray cast renderer.
intrinsic = o3d.camera.PinholeCameraIntrinsic(
640, 480, 572.4114, 573.5704, 325.2611, 242.0489)
renderer = m3d.pose_estimation.RayCastRenderer(intrinsic)
# Cast rays for single mesh with a associated pose.
ret = renderer.cast_rays([mesh], [pose])
depth = renderer.get_depth_map()
# Convert depth map into numpy array.
depth = depth.numpy()
# Get partial view point clouds of the mesh in the scene.
pcd = renderer.get_point_cloud()
# proximity extraction
pe = m3d.segmentation.ProximityExtractor(100)
ev = m3d.segmentation.DistanceProximityEvaluator(0.02)
index_list = pe.segment(pc, 0.02, ev)
# plane segmentation using iterative ransac
results = m3d.segmentation.segment_plane_iterative(pcd, 0.01, 100, 0.1)
# vis
# draw a pose represented as a axis
m3d.vis.draw_pose(vis, size=0.1)
# draw point clouds painted with red
m3d.vis.draw_geometry3d(vis, pcd, color=(1, 0, 0), size=3.0)
m3d.vis.draw_geometry3d(vis, mesh, color=(1, 0, 0))
m3d.vis.draw_geometry3d(vis, bbox, color=(1, 0, 0))
# logging
# the logging api is similar to open3d
# the VerbosityLevel is Info, Error, Debug and Warning
m3d.set_verbosity_level(m3d.VerbosityLevel.Error)
You can run c++ examples after finish build the library, which are inside /path/to/install/misc3d/bin
. The source code of examples are in examples/cpp
.
Misc3D is very open minded and welcomes any contribution. If you have any question or idea, please feel free to create issue or pull request to make Misc3D better. Please see this WIKI for contribution method.