Algomorph/NeuralTracking

Replace old CPU-based code with in the optimized CUDA code for mesh warping into the pipeline

Closed this issue · 0 comments

Overarching goal: gradually move towards a single representation of data in the code (to avoid so many CPU-->GPU transfers and in-memory copying). This will, at once, simplify maintenance, improve readability of code, and improve the performance of the pipeline.

Specific to this particular issue: in warp_field/graph.py, there are already two implementations of the "graph" data structure, DeformationGraphNumpy and DeformationGraphOpen3D. Both implement the same functionality, except the former performs everything on numpy arrays while the latter uses mosly Open3D Tensors (allowing it to function in CUDA memory and eliminating many RAM-->VRAM transfers).

In the meantime, apps/fusion/pipeline.py still uses the older DeformationGraphNumpy structure. What remains is to replace it with DeformationGraphOpen3D in the pipeline and throughout the rest of the code, then to test and ensure the functionality / correctness is not lost.

Subsequently, DeformationGraphNumpy should be eliminated from the code, and DeformationGraphOpen3D can be refactored to DeformationGraph.

Note: in the future, we might rewrite the motion graph warp field representation in C++ entirely, exposing it via pybind11.