LuJieJIE/RT_GestureRecognition

Real time gesture recognition using PyTorch

RT_GestureRecognition

This demo was trained on The 20BN-jester Dataset V1 (https://20bn.com/datasets/jester/v1)

Getting the demo to work:

Clone the repository. You might need to install some libs such as:

• PyTorch
• OpenCV2
• Numpy
• Pandas
• Torchvision

Run demo.py

Press Q to stop.

What is this network?

This is a custom NN using 2D and 3D CNN layers to achieve online video recognition. The model is trained on the Jester dataset, and adapted to the other dataset via trasnfer learning.

Getting the datasets used

The data used to train and test can be downloaded in the following links:

• Jester: https://20bn.com/datasets/jester
• nvGesture:https://research.nvidia.com/publication/online-detection-and-classification-dynamic-hand-gestures-recurrent-3d-convolutional
• EgoGesture: http://www.nlpr.ia.ac.cn/iva/yfzhang/datasets/egogesture.html

TO DO:

• Example of trainning
• Add pre-trained models for EgoGesture and nvGesture

Special thanks to @ahmetgunduz and @ms3001 for providing some of the code used here.

Citation

Please cite the following article if you use this code or pre-trained models:

@ARTICLE{9085286,
  author={F. {Brandolt Baldissera} and F. L. {Vargas}},
  journal={IEEE Latin America Transactions}, 
  title={A Light Implementation of a 3D Convolutional Network for Online Gesture Recognition}, 
  year={2020},
  volume={18},
  number={02},
  pages={319-326},
  doi={10.1109/TLA.2020.9085286}}