/EAO-SLAM

[IROS 2020] EAO-SLAM: Monocular Semi-Dense Object SLAM Based on Ensemble Data Association

Primary LanguageC++OtherNOASSERTION

EAO-SLAM

Related Paper:

  • Wu Y, Zhang Y, Zhu D, et al. EAO-SLAM: Monocular Semi-Dense Object SLAM Based on Ensemble Data Association[C]//2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2020: 4966-4973. [Paper] [Arxiv] [YouTube] [bilibili] [Project page].
  • Extended Work
    • Wu Y, Zhang Y, Zhu D, et al. Object SLAM-Based Active Mapping and Robotic Grasping[C]//2021 International Conference on 3D Vision (3DV). IEEE, 2021: 1372-1381. [Paper] [Arxiv] [Project page].
    • Robotic Grasping demo: YouTube | bilibili
    • Augmented Reality demo: YouTube | bilibili
  • If you use the code in your academic work, please cite the above paper.

1. Prerequisites

  • Prerequisites are the same as semidense-lines. If compiling problems met, please refer to semidense-lines and ORB_SLAM2.
  • The code is tested in Ubuntu 16.04, opencv 3.2.0/3.3.1, Eigen 3.2.1.

2. Building

chmod +x build.sh       
./build.sh

3. Examples

  • 3.0 We provide a demo that uses the TUM rgbd_dataset_freiburg3_long_office_household sequence; please download the dataset beforehand. The offline object bounding boxes are in data/yolo_txts folder.

  • 3.1 Object size and orientation estimation.

    • use iForest and line alignment: 
      ./Examples/Monocular/mono_tum LineAndiForest [path of tum fr3_long_office]
    • only use iForest: 
      ./Examples/Monocular/mono_tum iForest [path of tum fr3_long_office]
    • without iForest and line alignment: 
      ./Examples/Monocular/mono_tum None [path of tum fr3_long_office]

    Figure 1

  • 3.2 Data association

    • without data association: 
      ./Examples/Monocular/mono_tum NA [path of tum fr3_long_office]
    • data association by IoU only: 
      ./Examples/Monocular/mono_tum IoU [path of tum fr3_long_office]
    • data association by Non-Parametric-test only: 
      ./Examples/Monocular/mono_tum NP [path of tum fr3_long_office]
    • data association by our ensemble method: 
      ./Examples/Monocular/mono_tum EAO [path of tum fr3_long_office]

    Figure 1

  • 3.3 The full demo on TUM fr3_long_office sequence:

    ./Examples/Monocular/mono_tum Full [path of tum fr3_long_office]
    • If you want to see the semi-dense map, you may have to wait a while after the sequence ends.
    • Since YOLO (which was not trained in this scenario) made a lot of false detections at the start of the sequence, so we adopted a stricter elimination mechanism, which resulted in the deletion of many objects at the start.

4. Videos

5. Note

  • This is an incomplete version of our paper. If you want to use it in your work or with other datasets, you should prepare the offline semantic detection/segmentation results or switch to online mode. Besides, you may need to adjust the data association strategy and abnormal object elimination mechanism (We found the misdetection from YOLO has a great impact on the results).

6. Acknowledgement

Thanks for the great work: ORB-SLAM2, Cube SLAM, and Semidense-Lines.

  • Mur-Artal R, Tardós J D. Orb-slam2: An open-source slam system for monocular, stereo, and rgb-d cameras[J]. IEEE Transactions on Robotics, 2017, 33(5): 1255-1262. PDF, Code
  • Yang S, Scherer S. Cubeslam: Monocular 3-d object slam[J]. IEEE Transactions on Robotics, 2019, 35(4): 925-938. PDF, Code
  • He S, Qin X, Zhang Z, et al. Incremental 3d line segment extraction from semi-dense slam[C]//2018 24th International Conference on Pattern Recognition (ICPR). IEEE, 2018: 1658-1663. PDF, Code

7. Contact

@inproceedings{wu2020eao,
  title={EAO-SLAM: Monocular semi-dense object SLAM based on ensemble data association},
  author={Wu, Yanmin and Zhang, Yunzhou and Zhu, Delong and Feng, Yonghui and Coleman, Sonya and Kerr, Dermot},
  booktitle={2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
  pages={4966--4973},
  year={2020},
  organization={IEEE}
}

@inproceedings{wu2021object,
  title={Object SLAM-Based Active Mapping and Robotic Grasping},
  author={Wu, Yanmin and Zhang, Yunzhou and Zhu, Delong and Chen, Xin and Coleman, Sonya and Sun, Wenkai and Hu, Xinggang and Deng, Zhiqiang},
  booktitle={2021 International Conference on 3D Vision (3DV)},
  pages={1372-1381},
  year={2021},
  organization={IEEE}
}