/DGNN-Tensorflow

Unofficial Tensorflow implementation of the CVPR'19 paper "Skeleton-Based Action Recognition with Directed Graph Neural Networks".

Primary LanguagePython

DGNN-Tensorflow

An unofficial Tensorflow implementation of the paper "Skeleton-Based Action Recognition with Directed Graph Neural Networks" in CVPR 2019.

NOTE: Experiment results are not being updated due to hardware limits.

  • Paper: PDF
  • Code is based on DGNN-PyTorch: GitHub

Dependencies

  • Python >= 3.5
  • scipy >= 1.3.0
  • numpy >= 1.16.4
  • tensorflow >= 2.0.0

Directory Structure

Most of the interesting stuff can be found in:

  • model/dgnn.py: model definition of DGNN
  • data_gen/: how raw datasets are processed into numpy tensors
  • graphs/directed_ntu_rgb_d.py: graph definition for DGNN
  • main.py: general training/eval processes; etc.

Downloading & Generating Data

NTU RGB+D

  1. The NTU RGB+D dataset can be downloaded from here. We'll only need the Skeleton data (~ 5.8G).

  2. After downloading, unzip it and put the folder nturgb+d_skeletons to ./data/nturgbd_raw/.

  3. Generate the joint dataset first:

cd data_gen
python3 ntu_gen_joint_data.py

Specify the data location if the raw skeletons data are placed somewhere else. The default looks at ./data/nturgbd_raw/.

  1. Then, in data_gen/, generate the bone dataset:
python3 ntu_gen_bone_data.py
  1. Generate the motion data from joints/bones:
python3 ntu_gen_motion_data.py
  1. Generate the tfrecord files for motion and spatial data :
python3 ntu_gen_tfrecord_data.py

The generation scripts look for generated data in previous step. By default they look at ./data; change dir configs if needed.

Training

1st Stream: Spatial

To start training the network with the spatial stream, use the following command:

python3 main.py --config ./config/<dataset>/train_spatial.yaml

Here, <dataset> should be one of nturgbd-cross-subject, nturgbd-cross-view, or kinetics-skeleton depending on the dataset/task on which to train the model.

Note: At the moment, only nturgbd-cross-subject is supported. More config files will (hopefully) be added, or you could write your own config file using the existing ones for nturgbd-cross-subject.

2nd Stream: Motion

Similarly, to train on the motion stream data, do:

python3 main.py --config ./config/nturgbd-cross-subject/train_motion.yaml

and change the config file path for other datasets if needed.