stalomeow/StarRailMotionCapture

Motion capture for the character models of Honkai: Star Rail base on Unity and MediaPipe. Currently face only. (Do not need an iPhone)

StarRailMotionCapture

Motion capture for the character models of Honkai: Star Rail base on Unity and MediaPipe. Currently face only.

Known limitations

• Face capture only supports character models with facial skeleton, e.g., the models ripped directly from StarRail.

Protocol

Protocols between the server and the client are written in protobuf and compiled using my customized version of protoc.

Server

Developed with Python 3.10.

It captures motion data using your webcam or from a video file and sends them to clients.

Setup

All the requirements are listed in Server/requirements.txt.

Enter the Server folder and create a virtual environment. Then, run the command below.

pip install -r requirements.txt

Edit config

Edit Server/src/config.py.

Start the server

Run Server/src/main.py.

Client

Developed with Unity 2022.3.

Demo

A demo unity project is provided in Client/StarRailMotionCapture folder.

Create custom blend shape asset

1. Create asset using context menu.

   

2. Double click the asset to open the editor. Assign the face renderer of the character to Debug Skinned Mesh Renderer.

   

3. This editor supports Record/Preview mode like Unity's AnimationWindow, so you can use it as if you were using AnimationWindow.

4. Use the tools to increase efficiency.