/Pose-estimation-cv-system

Pose estimation cv system

Primary LanguagePython

Pose-estimation-for-weightlifters-cv-system

Pose estimation for weightlifters cv system

Purpose of the project

The goal of the project is to create a vision system capable of detecting the barbell and the athlete's foot during exercise in order to accurately analyze movement quality.

Used tools

  • darknet framework
  • yolov4/yolov4-tiny
  • google colabolatory
  • python
  • medipipe
  • opencv2
  • numpy

Training yolov4 and yolov4-tiny

Yolov4 achieved a mAp of about 99%.

Yolov4-tiny achieved a mAp of about 91%.

Pose detection module

The publicly available mediapipe library was used for position estimation. The library is based on neural networks, and the solutions implemented in it do not require training. The neural network outputs a set of points specifying the location of such body parts as eyes, mouth, nose, shoulders, elbows, hands, hip spikes, knees and feet and ankles. Using this information, the user is able to effectively determine joint angles and body alignments. A separate module called "plotter_3d.py" was created to visualize the output from the pose detector. It allows the user to save an animation containing an image and a 3d plot of the recorded person's pose after the detection is complete.

Unfortunately, pose detection during exercise proved to be impossible because the weight on the barbell covered parts of the exerciser's body, making the algorithm unable to perform detection.

Implementation

This project was implemented on the jetson-nano development kit. We evaluated it using gpu acceleration using OpenCV compiled with CUDA and Cudnn. The system achieved about 7 frames per second when detecting the foot and the barbell.

Video presentation

VIDEO

How to run this project

To run this project, create a virtualenv with python, preferably version 3.9, and install the necessary libraries using: pip install requirements.txt`. If you want to achive better performance compile OpenCV with CUDA, you can follow this tutorial: https://www.youtube.com/watch?v=YsmhKar8oOc. Now you can type python main.py`` in python and run the program with the desired flags defined in the table below.

Shortened argument flag Full flag of an argument Description
-s --save Saves data and graphs when recording is complete.
-vp --video-path Specifies the path to the recorded video. If the path is not specified, the detection will start from camera port 0.
-pd --pose-detector Performs pose detection using the mediapipe module.
-a --angles Computes angles at the joints specified in the constants file. (Module available only with the -pd argument.)
-b --barbell-detector Performs barbell and foot detection.
-f --fps Displays the number of fps.
-sa --save-animation Saves the animation after the detection. Specify the format. Available formats: mp4. (Module available only with the -pd argument.)
-d --display Displays a preview during recording.
-o --output-file Specifies the storage location and name of the csv file.