A web-based application analyses video source (eg. webcam) in realtime and counts each complete full motion of the persin in the video while they are doing exercises.
*I have never worked on any project relating to AI, so that my approach here is the best I can do.
I began the project with Tensorflow.js because it is designed to run on client side. I do not need to setup a complicated server and deal with server-client stuff.
As same as other AI frameworks, I have to feed Tensorflow.js a model and data that I want to predict.
If I do not want to do everything from scratch, there're pre-trained models created by others. Here, Tensorflow.js is hidden, I only work with simplified API provided by each pre-trained model.
I chose the pre-trained model Pose Dection. The model helps to make a prediction on each frame from a video source, each prediction returns a collection of keypoints (joints) with a corresponding confident score.
Because the pre-trained model does not directly counts a full motion but returns keypoints, I have to process the keypoints in order to determine whether a full motion is completed.
My very first step is to eliminate unnecessary prediction results, I call it noise reducing. I think it will be helpful for further steps.
What's next? The idea is to analyse the pattern of waves. The waves in the image above presenting my left shoulder position while I am doing rope jumping. When I jump up and down, the wave goes up and down, respectively.
For the implementation, I dont know haha. I will get back and think more. I jot everything down here mainly for myself, so later I will understand what is going on ~
Follow these steps:
-
Remove cache etc.
.cache,dist,node_modules -
Install dependencies.
yarn -
Run the app.
yarn watch -
The app runs at
localhost:1234.
Code base was copied from here https://github.com/tensorflow/tfjs-models/tree/master/pose-detection/demos/live_video