RGBDTracking is a SOFA plugin to register and track deformable objects using an RGB-D sensor in real-time.
The framework relies on a prior visual segmentation of the object in the image. The segmented point cloud is registered first in a rigid manner and then by non-rigidly fitting the mesh, based on the Finite Element Method to model elasticity, and on vision-based external forces exerted on the mesh.
The developped methods are described in the papers RAS 2017 and IROS 2015
Using Ubuntu up to 16-04 LTS as OS is preferable to benefit form all the features of the plugin (CUDA based segmentation).
The plugin is developed with the SOFA platform, we refer to the SOFA documentation for its up-to-date (master branch) installation (on Linux here).
Use gcc-5 preferably, apart from the optional custom CUDA installation which requires gcc-4.8 (see below how you can install CUDA and switch between compilers)
The plugin itself has the following dependencies:
Installation from [source] (https://docs.opencv.org/trunk/d7/d9f/tutorial_linux_install.html)
Required dependencies for PCL:
sudo apt-get install doxygen
sudo apt-get install mpi-default-dev openmpi-bin openmpi-common
sudo apt-get install libflann1.8 libflann-dev
sudo apt-get install libeigen3-dev
sudo apt-get install libboost-all-dev
sudo apt-get install libvtk6.2-qt4 libvtk6.2 libvtk6-dev
sudo apt-get install libqhull*
sudo apt-get install libusb-dev
sudo apt-get install libgtest-dev
sudo apt-get install git-core freeglut3-dev pkg-config
sudo apt-get install build-essential libxmu-dev libxi-dev
sudo apt-get install libusb-1.0-0-dev graphviz mono-complete
sudo apt-get install libproj-dev
sudo apt-get install qt-sdk openjdk-8-jdk openjdk-8-jre
sudo apt-get install phonon-backend-gstreamer
sudo apt-get install phonon-backend-vlc
Installation
Sources for PCL 1.9 can be found here https://github.com/PointCloudLibrary/pcl/releases/tag/pcl-1.9.1
Unzip and then:
cd pcl-pcl-1.9.1
mkdir build
cd build
cmake -DCMAKE_BUILD_TYPE=None -DBUILD_GPU=ON -DBUILD_apps=ON -DBUILD_examples=ON ..
make
sudo make install
sudo apt-get install libfreeimage3 libfreeimage-dev
Install ViSP from source and link it with OpenCV 3
Follow these instructions for the dependencies for ViSP
Sources for ViSP 3.0.0 or later can be found here https://github.com/lagadic/visp/releases
Unzip and then:
cd visp-3.0.0
mkdir build
cd build
ccmake -DUSE_apps=ON ..
set the OPENCV_DIR in the ccmake to the build or install directory of OpenCV 3
make
sudo make install
Cuda (optional, if you have an Ubuntu > 16.04 LTS or no available NVIDIA Graphic card, directly go to step 2)
Cuda (<= 7.0) is required to enable some features for the CUDA based segmentation (a CUDA free CPU implementation of the segmentation method, based on OpenCV, is also availble)
For Ubuntu 16.04 and Cuda 7.0:
First uninstall current Nvidia driver version
sudo apt-get --purge remove nvidia-*
No need to create an xorg.conf file. If you have one, remove it (assuming you have a fresh OS install).
sudo rm /etc/X11/xorg.conf
Blacklist the "nouveau" driver
echo -e "blacklist nouveau\nblacklist lbm-nouveau\noptions nouveau modeset=0\nalias nouveau off\nalias lbm-nouveau off\n" | sudo tee /etc/modprobe.d/blacklist-nouveau.conf
echo options nouveau modeset=0 | sudo tee -a /etc/modprobe.d/nouveau-kms.conf
sudo update-initramfs -u
Install Nvidia driver 384.90
sudo apt-get install nvidia-384-dev
Cuda 7.0 supports up to gcc-4.8 compiler. Install it, assuming you have gcc-5 already installed and used to install the previously mentioned libraries:
sudo apt-get install gcc-4.8 g++-4.8
sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-5 10
sudo update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-5 10
sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-4.8 20
sudo update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-4.8 20
You can then manually switch with gcc-5
sudo update-alternatives --config gcc
sudo update-alternatives --config g++
Download CUDA toolkit 7.0 (officially supported by Ubuntu 14.04) here as a runfile
Go to tty (Ctrl + Alt + F5) and do:
sudo service lightdm stop
In your Downloads folder do:
chmod +x cuda_7.0.28_linux.run
sudo ./cuda_7.0.28_linux.run --no-opengl-lib
Answers to cuda installation requests:
Do you accept the previously read EULA? (accept/decline/quit): accept
You are attempting to install on an unsupported configuration. Do you wish to continue? ((y)es/(n)o) [ default is no ]: yes
Install NVIDIA Accelerated Graphics Driver for Linux-x86_64 352.39? ((y)es/(n)o/(q)uit): no
Install the CUDA 7.0 Toolkit? ((y)es/(n)o/(q)uit): yes
Enter Toolkit Location [ default is /usr/local/cuda-7.0 ]: press enter or specify a path
Do you want to install a symbolic link at /usr/local/cuda? ((y)es/(n)o/(q)uit): y
Install the CUDA 7.0 Samples? ((y)es/(n)o/(q)uit): y
Enter CUDA Samples Location [ default is /home/user ]: presse enter or specify a path
Reboot:
sudo reboot
Set environment path variables in .bashrc:
export PATH=/usr/local/cuda-7.0/bin:$PATH
export LD_LIBRARY_PATH=/usr/local/cuda-7.0/lib64:$LD_LIBRARY_PATH
To check your installation, compile the CUDA Samples
cd home/user/NVIDIA_CUDA-7.0_Samples
make
cd bin/x86_64/linux/release/
./deviceQuery
The configurations and versions of your NVIDIA graphic card and drivers and CUDA toolkit should then appear.
Switch back to gcc-5
Edit the SOFA plugin CMakeLists to add the plugin gedit /home/.../sofa/master/src/applications/plugins/CMakelists.txt
and add the line sofa_add_plugin(RGBDTracking RGBDTracking)
In your build directory
cd /home/.../sofa/master/build
ccmake ../src
In the cmake GUI, activate the plugin 'RGBDTracking' and the plugin 'image', set 'OpenCV_DIR' to the build directory of your compiled OpenCV, and if CUDA 7.0 is installed, set 'CUDA_HOST_COMPILER' to /usr/bin/g++-4.8
Configure ang generate your cmake and run make.