Rigging.js

Rigging.js is an open-source react.js application, that takes the keypoints generated by the face mesh tensorflow.js model, then map the movement that the person is doing in front of the camera into a 3d model. Any model downloaded from https://www.mixamo.com/ can be used.

Build And Run

git clone https://github.com/haruiz/RiggingJs.git
cd RiggingJs
npm install
npm start

Try with different models

As mentioned above, any model downloaded from https://www.mixamo.com/ can be used. These models are freely available.

Roadmap

Automatic Rigging and Animation of 3D Characters: using the current state of the art deep learning models.
3D expression animation using facemesh model
Record animation

You can see the detailed roadmap here.

How to contribute:

Feel free to send a pull request

Third party libraries:

Inspiration taken from:

Pose Animator 2: Pose Animator takes a 2D vector illustration and animates its containing curves in real-time based on the recognition result from PoseNet and FaceMesh. It borrows the idea of skeleton-based animation from computer graphics and applies it to vector characters.
PoseNet Model: This package contains a standalone model called PoseNet, as well as some demos, for running real-time pose estimation in the browser using TensorFlow.js.
BodyPix: This package contains a standalone model called BodyPix, as well as some demos, for running real-time person and body part segmentation in the browser using TensorFlow.js.
MediaPipe Facemesh: MediaPipe Facemesh is a lightweight machine learning pipeline predicting 486 3D facial landmarks to infer the approximate surface geometry of a human face (paper).