This repository contains software programs for image-based registration, system integration and navigation tasks relating to orthopedic surgical robot applications. The repository is developed based on xReg, following its general structure and routines. For more information of xReg, please visit the wiki for descriptions on the use of the library and executable programs. This repository forks the complete library support of xReg. The compilation of xReg and thirdparty libraries can be done using stand-alone clone of this repository.
This software was developed with support from Dr. Robert Grupp, while conducting research under the supervision of Profs. Mehran Armand, Russell Taylor and Mathias Unberath within the Laboratory for Computational Sensing and Robotics at Johns Hopkins University.
- Registration:
- Efficient ray casters for 2D/3D registration and visualization:
- Image similarity metrics for driving 2D/3D registrations:
- Sum of squared differences (SSD) (CPU and OpenCL)
- Normalized cross correlation (NCC) (CPU and OpenCL)
- NCC of Sobel Gradients (Grad-NCC) (CPU and OpenCL)
- Gradient orientation (GO) (CPU)
- Gradient difference (Grad-Diff) (CPU)
- Patch-wise NCC (Patch-NCC) (CPU and OpenCL)
- Patch-wise Grad-NCC (Patch-Grad-NCC) (CPU and OpenCL)
- Boundary contour distance (CPU)
- Extendable common interface (CPU, OpenCL, and more)
- Various optimization strategies for 2D/3D intensity-based registration:
- Regularizers for 2D/3D intensity-based registration:
- Rotation and translation magnitudes
- Euler decomposition magnitudes
- Rotation and translation magnitudes of relative pose between multiple objects
- Relative pose difference from nominal AP pose
- Landmark re-projection distances
- Heuristics for automatic global pelvis registration
- Combination of regularizers
- Extendable common interface
- Pipeline for chaining together 2D/3D registrations (intensity-based and feature-based) for solving registration problems with multiple-resolutions and views
- Perspective-n-Point (PnP) solvers (paired point 2D/3D)
- Paired Point 3D/3D
- 3D Point Cloud to 3D Surface ICP
- Mesh Processing:
- Image/Volume Processing:
- Interpolation of non-uniform spaced slices
- Image processing operations leveraging lower-level ITK and OpenCV routines
- Conversion of Hounsfield units (HU) to linear attenuation
- Poisson noise
- Image intensity log transform
- Piecewise rigid volume warping using label maps
- Numerical Optimization:
- Configurable line search implementation
- Implementations of several derivative-free methods, including Differential Evolution, Simulated Annealing, and Particle Swarm Optimization
- Wrapper around C implementation of CMA-ES optimization
- Suite of test objective functions
- Geometric Primitives and Spatial Data Structures
- KD-Tree for points or surfaces of arbitrary dimension
- Primitives with support for intersection, etc.
- Fitting primitives to data (circle, plane) with robustness to outliers
- Spatial Transformation Utilities:
- Rotation and rigid transformation utilities, including lie group/algebra routines
- Perspective projection (3D to 2D)
- Calculation of anatomical coordinate frames
- Point cloud manipulation
- Visualization
- File I/O:
- Common DICOM fields
- DICOM files from Siemens CIOS Fusion C-arm
- Helpers for HDF5 reading/writing
- Various mesh formats
- Various image/volume formats (via ITK)
- 3D Slicer annotations, FCSV and ACSV
- Comma separated value (CSV) files
- Basic Math Utilities:
- Basic statistics
- Distribution fitting
- Uniformly distributed N-D unit vector sampling
- Common interface for probability densities and instances for common distributions:
- General/Common:
- Hip Surgery:
- Guessing labels of bones from segmentation volumes
- Planning and modeling of osteotomies
- Visualization of osteotomies in 3D
- Modeling of surgical objects, such as screws and K-wires
- Support for simulated data creation, including randomized screw and K-wire shapes and poses, and volumetric data incorporating osteotomies, repositioned bones, and inserted screws and K-wires
Some of the capabilities provided by individual programs contained with the apps directory include:
- Image I/O:
- Mesh processing:
- Basic point cloud operations:
- Registration
- General utilities for projection data:
- Advanced visualization of projective geometry coordinate frames with a scene of 3D objects
- Remap and tile projection data for visualization
- Tool for creating movie replays of 2D/3D registration processing
- Extract projection into NIFTY format (.nii/.nii.gz)
- Insert landmarks (FCSV) into HDF5 projection data
- Hip Surgery: Periacetabular Osteotomy (PAO)
- Osteotomy planning and modeling
- Osteotomy 3D visualization
- Randomized simulation of fragment adjustments
- Volumetric modeling of fragment adjustments
- Volumetric modeling of fragment fixation using screws and K-wires
- Creation of simulated fluoroscopy for 2D/3D registration experiments
- Examples of 2D/3D, fluoroscopy to CT, registration
Although the following capabilities currently only exist in an internal version of the xReg software, they will be incorporated into this repository at a future date:
- Executable for running a multiple-view/multiple-resolution 2D/3D registration pipeline defined using a configuration file
- Intraoperative reconstruction of PAO bone fragments
- Utilities for creation and manipulation of statistical shape models
- Shape completion from partial shapes and statistical models
- More point cloud manipulation utilities
- Python bindings, conda integration
- And more...
- C++ 11 compatible compiler
- External libraries (compatible versions are listed):
- OpenCL (1.x) (typically provided with your graphics drivers or CUDA SDK)
- Intel Threading Building Blocks (TBB) (20170919oss)
- Boost (header only) (1.65)
- Eigen3 (3.3.4)
- fmt (5.3.0)
- NLOpt (2.5.0)
- ITK (4.13.2)
- VTK (7.1.1)
- OpenCV (3.2.0)
- ViennaCL (1.7.1)
- Optional: ffmpeg is used for writing videos when it is found in the system path. The OpenCV video writer is used if ffmpeg is not available.
A standard CMake configure/generate process is used. It is recommended to generate Ninja build files for fast and efficient compilation. An example script for building all dependencies (except OpenCL) and the xReg repository is also provided here. The docker directory demonstrates how Docker may be used to build the software.
Development of this software results in the following publication references:
C. Gao et al., "Fiducial-Free 2D/3D Registration for Robot-Assisted Femoroplasty," in IEEE Transactions on Medical Robotics and Bionics, vol. 2, no. 3, pp. 437-446, Aug. 2020, doi: 10.1109/TMRB.2020.3012460.
----------------------------------------------------------------------
@ARTICLE{9151197, author={C. {Gao} and A. {Farvardin} and R. B. {Grupp} and M. {Bakhtiarinejad} and L. {Ma} and M. {Thies} and M. {Unberath} and R. H. {Taylor} and M. {Armand}}, journal={IEEE Transactions on Medical Robotics and Bionics}, title={Fiducial-Free 2D/3D Registration for Robot-Assisted Femoroplasty}, year={2020}, volume={2}, number={3}, pages={437-446}, doi={10.1109/TMRB.2020.3012460}}
Cong Gao, Robert B. Grupp, Mathias Unberath, Russell H. Taylor, Mehran Armand, "Fiducial-free 2D/3D registration of the proximal femur for robot-assisted femoroplasty," Proc. SPIE 11315, Medical Imaging 2020: Image-Guided Procedures, Robotic Interventions, and Modeling, 113151C (16 March 2020); https://doi.org/10.1117/12.2550992
----------------------------------------------------------------------
@inproceedings{10.1117/12.2550992,
author = {Cong Gao and Robert B. Grupp and Mathias Unberath and Russell H. Taylor and Mehran Armand},
title = {{Fiducial-free 2D/3D registration of the proximal femur for robot-assisted femoroplasty}},
volume = {11315},
booktitle = {Medical Imaging 2020: Image-Guided Procedures, Robotic Interventions, and Modeling},
editor = {Baowei Fei and Cristian A. Linte},
organization = {International Society for Optics and Photonics},
publisher = {SPIE},
pages = {350 -- 355},
keywords = {2D/3D Registration, Femur Registration, X-ray Navigation, Femoroplasty},
year = {2020},
doi = {10.1117/12.2550992},
URL = {https://doi.org/10.1117/12.2550992}
}