Realtime Isolated ASL Gesture Sequence Recognition

Experiment with MediaPipe to extract and display landmarks for Hands and Pose in RealTime use a Video Feed (Webcam)

Setting up your Environment

To run the code locally, you'll need to install and setup a few things:

Python 3 (if you don't have a recent version of Python, grab one here. We've tested on Python 3.10)
Poetry (dependency manager for Python - read the installation instructions here)
Git command line tools (the git command). Get these from the Git website if needed.

Cloning this Repository

At the terminal, clone the repository to your local machine:

git clone https://github.com/bsbodden/ASL_with_GNNs_Sam-Bodden_Brian.git

Then, change directory into the repository folder:

cd ASL_with_GNNs_Sam-Bodden_Brian

Installing Python Dependencies

We're using the Poetry tool to manage Python virtual environments and dependencies. Install the dependencies that this workshop uses with the following command:

poetry install

Running the Experiments

Pose Landmarker Demo

Based on https://developers.google.com/mediapipe/solutions/vision/pose_landmarker/python

poetry run python PoseLandmarkerDemo.py

Hand Landmarker Demo

Based on https://developers.google.com/mediapipe/solutions/vision/hand_landmarker/python#live-stream

poetry run python HandLandmarkerDemo.py

Hand Landmarker Finger Counting Demo

Reverse engineering of this Video https://www.youtube.com/watch?v=p5Z_GGRCI5s (this was bad code and for an ancient version of MediaPipe)

poetry run python HandLandmarkerFingerCounterDemo.py

Hand Gesture Classification Demo

Reverse engineering of this Video https://www.youtube.com/watch?v=p5Z_GGRCI5s (this was bad code and for an ancient version of MediaPipe)

poetry run python HashGestureClassificationDemo.py

Face Landmarker Demo

Reverse engineering of this Video https://www.youtube.com/watch?v=p5Z_GGRCI5s (this was bad code and for an ancient version of MediaPipe)

poetry run python FaceLandmarkerDemo.py

LandmarksPlayer

A tkinter app to visualize the data set from https://www.kaggle.com/competitions/asl-signs/data - download the zip file and unzip it somewhere in your computer, say ~/Desktop/asl-signs then make a copy of .env-examples, rename it to .env and change the environment property ASL_SIGNS_BASE_DIRECTORY to point to the asl-signs folder.

poetry run python LandmarksPlayer.py

To quit any of the demos, press "q"