/hand_gesture_detect

using CNN and LSTM to build static and dynamic hand gesture detect

Primary LanguagePython

hand_gesture_detect

Description

Use python tensorflow Keras to build CNN and LSTM model, use them to predict static and dynamic hand gesture.

Data preparation

use file change_img_to_YCrCb.py to change pictures into black-and-white pictures, As it is more simple to identify through black-and-white picture.

using CNN

Using CNN to identify static hand gesture

  • neural network structure:

Result

Get test accuracy up to 0.75.

  • testing result:

  • prediction compare to real label:

Because of using black-and-white picture, the prediction quality isn't good for some similar pictures (see example below):

using LSTM

Using LSTM to identify dynamic hand gesture.

  • neural network structure:

make a series of data

Put 10 pictures in a group and label them. Add "swapping left" and "swapping right" gesture (see pictures below), labeled as 11 and 12.

  • swapping left:

  • swapping right:

Result

Get test accuracy up to 0.6538.

  • testing result:

  • prediction compare to real label:

LSTM model can predict dynamic gesture well (label 11 and 12), but would made more error on identifying static gesture.