TanCK97/robot-handover

Perception for human robot handover

Human-Robot Handover

Requirements

• detectron2 (Follow INSTALL.md to install detectron2)
• mtcnn

Pretrained models

Download the pretrained models for object detection, head pose estimation and MLP. Place them in ./pretrained-weights.

Custom Dataset

The handover dataset used to obtain the pretrained weights can be found here.

Run inference

python3 run_inference.py \
        --cfg-keypoint ./configs/keypoint_rcnn_R_101_FPN_3x.yaml \
        --cfg-object ./configs/object_faster_rcnn_R_101_FPN_3x.yaml \
        --obj-weights ./pretrained-weights/Apple_Faster_RCNN_R_101_FPN_3x.pth \
        --video-input [VIDEO_INPUT] \
        --output [OUTPUT] \
        --mlp

Generate a JSON file for MLP training

python3 utils/json_utils.py --json-path [JSON_FOLDER] --csv-path [classes.csv] \
                      --output-json sample_robot.json

Train MLP network

python3 train_MLP_localize.py --json-path [JSON_FILE] --weights-path [PATH_TO_WEIGHTS]