/SysMocap

A real-time motion capture system for 3D virtual character animating.

Primary LanguageJavaScriptMozilla Public License 2.0MPL-2.0

banner

SysMocap

GitHub Actions release forks stars

English Version | 中文版本

A cross-platform real-time video-driven motion capture and 3D virtual character rendering system for VTuber/Live/AR/VR.

Available for Windows, macOS (packaged) & Linux (source code only)

Download Now (zipped, without installation)

(This is a multi-language software, including English. )

Highlights

🌟 Beautiful GUI with Material Design 3 Color System

GUI

🌟 Easy to used. You can import 3D models just with drags.

add-model-drag.mp4

🌟 Support WebXR API on Mocap Forwarding (HTTPS only)

webxr-ar-demo.mp4

🌟 Model viewer with bones & dressing controller

Model viewer

🌟 Support OBS live-streaming

OBS

🌟 Support full-body motion capture

Full-body

Full-body-animotion

🌟 Support Auto Skeleton Detection for All VRM files and Mixamo Format FBX files

fbx-mixamo-animotion

🌟 Support Any Skeleton Structure with Manual Mapping

bdd-animotion

🌟 Does not require a discrete graphics card and runs smoothly even on eight-year-old computers (i7-4790k/GTX770/16G RAM)

🌟 Powered by Mediapipe and Kalidokit with Web Technologies

More Effect Demonstration

🌟 Facial

Facial-animotion

🌟 Half-body

Half-body-animotion

🌟 Half-body with Hands

Half-body with Hands animotion

🌟 Full-body

Full-body-animotion

System architecture

System architecture

How to use

Run on your computer from source code (need lastest Node.js):

git clone https://github.com/xianfei/SysMocap.git
cd SysMocap
npm i
npm start

Bugs

  1. On Windows platform, "Use Discrete Graphics on Dual GPU Laptop" and "Mocap Data Forward" can not use at same time.

  2. Camera controller only support VRM

  3. Forwarding only support VRM

Notice

  1. HTTP & HTTPS will use same port in Mocap Data Forward.

Required Skeleton Node in glTF/glb/FBX Model File for Macap:

(If not same as above, you need rebind them manually.)

  • Hips (Main Node, both Position and Rotation. Ratation only for other nodes)

  • Neck

  • Chest

  • Spine

  • RightUpperArm

  • RightLowerArm

  • LeftUpperArm

  • LeftLowerArm

  • LeftUpperLeg

  • LeftLowerLeg

  • RightUpperLeg

  • RightLowerLeg

Development progress

To-Do

  • Settings page and global settings utils

  • Add play/pause button and progress bar when mocap from video

  • Support bones binding for glTF/glb

  • Support rendering glTF/glb model

  • Support binding when bones' name is non-uniformed

  • Model library add user's custom 3D model

  • Live plug-in / interface for Open Broadcast Software

  • Output video ( using such as libffmpeg )

  • Support per-frame rendering without drop frame

  • Support c-s architecture for online video mocap ( on cloud )

  • Support Material Designed 3 Color System (color picking)

  • Mocap data forwarding via network

  • Adapt for Linux and macOS

Thanks

You can see all dependencies in package.json