UmithanGoktolga/Intention_Prediction

Intention prediction

An introduction to intention prediction scenario and data set.

Experiment scenario

The experiment is about a human participant pitching a ball toward a robot (represented by a target area in the figure). The target area is divided into 9 grids which represent the 9 targets that the ball can hit. We expect the robot to be able to predict the intention of the participant (i.e., which of the nine targets the participant is pitching the ball to).

Single-participant Data set

A data set of human intentions behind human actions. This data set is built from one human priticipant. The dataset contains RGB images, RGB-D images, and skeleton data of human actions.

The data set is free to use for non-commercial purposes.

• RGB images: RGB-train (extraction code: 527f) RGB-test (extraction code: aw3x)

• RGB-D images: RGB-D-train (extraction code: 3ekq) RGB-D-test (extraction code: 8uaw)

• Skeleton data: Skeleton-train (extraction code: u485) Skeleton-test (extraction code: xk5p)

Multi-participant Data set

A data set of human intentions behind human actions. This data set is built from multiple human priticipants. The dataset contains RGB images, RGB-D images, and skeleton data of human actions.

The data set is free to use for non-commercial purposes.

• RGB images: RGB-part1 RGB-part2 RGB-part3 RGB-part4 RGB-part5

• RGB-D images: RGB-D-part1 RGB-D-part2 RGB-D-part3

• Skeleton data: Skeleton

If this data set is useful for your research, please refer to our paper:

Shengchao Li, Lin Zhang, and Xiumin Diao - Deep-Learning-Based Human Intention Prediction Using RGB Images and Optical Flow (BibTeX)

Others

1. data_acquire: capture 3 sec video by Kinect V2 under Matlab and save the data.

2. dataset_utils:

• travaltes_slicing.py slices dataset to train/validation/test by proportion of 6/2/2.
• create_pitch_tf_record.py create tfrecord shards.

3. pitch2d_main.py training and evaluation.