/Segmentation-to-World-Coordinate-by-FCN-and-Camera-Calibration

Application of FCN on grasping task, get output of world coordinate point to which robot arm would go.

Primary LanguagePythonMIT LicenseMIT

Segmentation to World Coordinate by FCN and Camera Calibration

  • Grasping Project for Korea University Robot Club, KAsimov.
  • Segmentation by Fully Convolutional Network.
  • Backbone by ResNet 18 pretrained on ImageNet. (Deprecated)
  • Backbone by VGG 16 pretrained on ImageNet.
  • Full model is Pretrained on PASCAL VOC 2012.
  • Fine tuned on Own dataset, for 3 classes.

Dependency and Commands to Run

To start realsense camera package, do this command.

source /opt/ros/noetic/setup.bash && source ~/catkin_ws/devel/setup.bash
roslaunch realsense2_camera rs_camera.launch align_depth:=true

To start segmentation_to_world package, do this command.

source /opt/ros/noetic/setup.bash && source ~/catkin_ws/devel/setup.bash
cd src
python3 cam.py

Result of Metric

  • test on PASCAL VOC 2012 validation set
    • Accuracy : 86.19%
    • Mean IoU : 42.30%

Reference for this repo

  • Intrinsic camera calibration
  • Extrinsic camera calibration

Demo Videos

Image Image Image