SysMocap

English Version | 中文版本

A cross-platform real-time video-driven motion capture and 3D virtual character rendering system for VTuber/Live/AR/VR.

Available for Windows, macOS (including Apple Silicon & Intel) & Linux (source code only)

Download Now

(This is a multi-language software, including English and Chinese. )

Highlights

🌟 Beautiful GUI with Material Design 3 Color System, with Dark Mode supported.

🌟 Easy to used. You can import 3D models just with drags.

add-model-drag.mp4

🌟 Support WebXR API on Mocap Forwarding (HTTPS only)

webxr-ar-demo.mp4

🌟 Model viewer with bones & dressing controller

🌟 Support OBS live-streaming

🌟 Support full-body motion capture

🌟 Support Auto Skeleton Detection for All VRM files (including VRM 0.x and VRM 1.0) and Mixamo Format FBX files

🌟 Support Any Skeleton Structure with Manual Mapping

🌟 You can use VRoid Studio to create your avatar very easily. Supporting VRM 0.x & 1.0!

More Effect Demonstration

🌟 Facial

🌟 Half-body

🌟 Half-body with Hands

🌟 Full-body

System architecture

How to Download prebuilt package (Windows & macOS):

Download Now

1. Windows Portable Version: Simply extract the archive and run SysMocap.exe.

SysMocap-Windows-x64-<version>.7z: Windows 64-bit version, suitable for x86_64 processors (AMD, Intel, etc.) running 64-bit Windows 10 & 11.
SysMocap-Windows-arm64-<version>.7z: Windows ARM 64-bit version, suitable for ARM64 processors (such as Qualcomm Snapdragon) running 64-bit Windows 10 & 11.

2. Windows Installer: Double-click to install.

SysMocap-Windows-x64-installer-<version>.msi: Windows 64-bit version, suitable for x86_64 processors (AMD, Intel, etc.) running 64-bit Windows 10 & 11.
SysMocap-Windows-arm64-installer-<version>.msi: Windows ARM 64-bit version, suitable for ARM64 processors (such as Qualcomm Snapdragon) running 64-bit Windows 10 & 11.

3. macOS DMG Image: Drag SysMocap.app to the Applications folder.

SysMocap-macOS-x64-<version>.dmg: Suitable for Apple computers with Intel chips and Hackintosh devices, running macOS 10.15 or later.
SysMocap-macOS-arm64-<version>.dmg: Suitable for Apple computers with M series chips (Apple Silicon).

Note for user on macOS:

You need set Gatekeeper to Anywhere in System Settings (using sudo spctl --master-disable)
If you got “SysMocap” is damaged and can’t be opened. You should move it to the Trash. Please run sudo xattr -r -d com.apple.quarantine /Applications/SysMocap.app in your terminal

How to run from source code (need lastest Node.js):

git clone https://github.com/xianfei/SysMocap.git
cd SysMocap
npm i
npm start

Notice

HTTP & HTTPS will use same port in Mocap Data Forward.

Required Skeleton Node in glTF/glb/FBX Model File for Macap:

(If not same as above, you need rebind them manually.)

Hips (Main Node, both Position and Rotation. Ratation only for other nodes)
Neck
Chest
Spine
RightUpperArm
RightLowerArm
LeftUpperArm
LeftLowerArm
LeftUpperLeg
LeftLowerLeg
RightUpperLeg
RightLowerLeg

Star History

Thanks

You can see all dependencies in package.json

google/mediapipe/Holistic for Mocap
kalidokit for Calulate Mocap Data
electron and Vue.js for GUI Framework
Material color utilities for Color Picking

Cite

@INPROCEEDINGS{9974484,
  author={Song, Wenfeng and Wang, Xianfei and Gao, Yang and Hao, Aimin and Hou, Xia},
  booktitle={2022 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct)}, 
  title={Real-time Expressive Avatar Animation Generation based on Monocular Videos}, 
  year={2022},
  volume={},
  number={},
  pages={429-434},
  doi={10.1109/ISMAR-Adjunct57072.2022.00092}}