- Ensure that you have torch, torchvision, opencv, numpy and Pillow libraries installed.
- change directory to the
Hand_Detection_YOLOv5
folder. - run
git clone https://github.com/ultralytics/yolov5.git
- Download the FreiHAND dataset from https://lmb.informatik.uni-freiburg.de/resources/datasets/FreihandDataset.en.html
- Extract the zip file into the
Hand_Pose_Estimation_2D
folder and name it as "FreiHAND_pub_v2"
For training of the YOLOv5 model, follow the steps below:
- Change directory to the local repository of yolov5 which we have cloned earlier
- run
python train.py --data <path to yaml file> --batch <number of batches> --epochs <number of epochs> --weights yolov5m.pt
For training of the Hand Pose Estimation model, follow the steps below:
- Change directory to
Hand_Pose_Estimation_2D
folder - Run all the cells within
Train.ipynb
(takes approximately 6 hours for training) - Evaluate the model performance by running cells in
Inference.ipynb
To use the final model that incorporates both object detection and handpose estimation techniques, perform the following steps:
- Change directory to
Hand_Detection_YOLOv5
folder - Run cells in
demo.ipynb
To use the final model that leverages on MediaPipe, perform the following steps:
- change directory to
mediapipedrums
folder - run
demo.py
It is recommended that you carry out these steps in Visual Studio Code. In the command line terminal, ensure that you are in the 50.035CV-C01-Team10-Visual-Interactive-Game folder. Then, execute the following steps:
- Create a virtual environment
- POSIX:
python3 -m venv venv
- Windows:
py -3 -m venv venv
- Activate the virtual environment
- POSIX:
. venv/bin/activate
- Windows:
venv\Scripts\activate
-
Install dependencies in the virtual environment
pip install -r requirements.txt
-
Change directory to <~/mediapipedrums> folder
cd mediapipedrums
-
Run demo.py file
python demo.py