/mask-ukf

Instance Segmentation Aided 6D Object Pose and Velocity Tracking using an Unscented Kalman Filter

Primary LanguageC++GNU General Public License v2.0GPL-2.0

MaskUKF

An Instance Segmentation Aided Unscented Kalman Filter for 6D Object Pose and Velocity Tracking

Frontiers in Robotics and AI
Paper | Video

DOI CI badge

Reproducing the experiments

We support running the experiments via the provided Docker image.

  1. Pull the docker image: 
    docker pull ghcr.io/robotology/mask-ukf:latest
  2. Launch the container: 
    docker run -it --rm --user user --env="DISPLAY" --net=host ghcr.io/robotology/mask-ukf:latest
  3. Clone and build the project: 
    git clone https://github.com/robotology/mask-ukf.git
cd mask-ukf
mkdir build && cd build
cmake ../
make
  4. Download and unzip the accompanying data DOI and the YCB-Video model set: 
    cd /home/user/mask-ukf
bash misc/download_accompanying_data.sh
bash misc/download_ycb_models.sh
  5. Run the experiments (optional): 
    cd /home/user/mask-ukf
bash test/test_all.sh

    The accompanying data contains the pre-evaluated results. If desired, the results can be re-evaluated using the above command.

  6. Run the evaluation: 
    cd /home/user/mask-ukf
bash evaluation/evaluate_<mask_set>_<metric>_<algorithm>.sh

    <mask_set> can be mrcnn (Mask R-CNN) or posecnn (PoseCNN), <metric> can be add_s (ADD-S) or rmse (RMSE) and <algorithm> can be empty (MaskUKF), icp (ICP) or densefusion (DenseFusion).

  7. Visualize the results: 
    cd /home/user/mask-ukf
python3 evaluation/renderer/renderer.py --algorithm <algorithm> --mask_set <mask_set> --object <object_name> --video_id <video_id>

    <algorithm> can be mask-ukf (MaskUKF), icp (ICP) or dense_fusion (DenseFusion), <mask_set> is as above, <object_name> is e.g. 002_master_chef_can and <video_id> is the YCB-Video video id, e.g. 0048.

In order to run the visualizer it could be required to temporarily execute xhost + in a console outside of Docker in order to allow the container accessing the X server facilities. The command can be run even after the container has been already launched.

If you want to install the repository manually, please refer to the recipe contained in the Dockerfile. Please be aware that the results might differ if unsupported versions of the dependencies are used.

Citing MaskUKF

If you find the MaskUKF code useful, please consider citing the associated publication:

@ARTICLE{10.3389/frobt.2021.594583,
AUTHOR={Piga, Nicola A. and Bottarel, Fabrizio and Fantacci, Claudio and Vezzani, Giulia and Pattacini, Ugo and Natale, Lorenzo},
TITLE={MaskUKF: An Instance Segmentation Aided Unscented Kalman Filter for 6D Object Pose and Velocity Tracking},
JOURNAL={Frontiers in Robotics and AI},
VOLUME={8},
PAGES={38},
YEAR={2021},
URL={https://www.frontiersin.org/article/10.3389/frobt.2021.594583},
DOI={10.3389/frobt.2021.594583},
ISSN={2296-9144}
}

and/or the repository itself by pressing on the Cite this respository button in the About section.

Maintainer

This repository is maintained by:
@xenvre