noname0310/babylon-mmd

Whether other character models can load vmd animation format

Closed this issue · 3 comments

spencer17x commented
Whether other character models can load vmd animation format
noname0310 commented

Because mmd models have their own bone structure, using vmd motions in other models requires retargeting, and even if retargeted, it is not possible to load a vmd model into a humanoid skeleton or something similar normally because "IK" and "append transform" do not exist in non-mmd models.

A theoretically possible way:

Loading VMD animations into other skeletons requires some understanding of the MMD runtime.

MmdRuntime.createMmdModel You must first call this method to create a Skeletal Mesh that is not an mmd model to be manipulated by the MMD Runtime. To do this, the mesh you want to load will need to be modified to follow the specifications of the MmdMesh interface.

In the process, you will need to create some metadata yourself, which is really only possible with a full understanding of the standard bone structure of modern MMD.

MmdModel.addAnimation takes a retargeting map as its second parameter, so you can utilize it to fully play the vmd animation.

  • I'll soon provide a way to play animations using BabylonJS's Animation container, but even with this, loading motion can give strange results if the Mesh is not under the control of the MmdRuntime.

  • It may be possible to provide the ability to virtualize Skeletal Meshes for humanoid structures to behave as if they were MMD bones, but I'm not sure if this is possible. This is something that will be considered for future implementation.

noname0310 commented

The easiest way is to simply use a solution that converts your model to PMX format.

noname0310 commented

As of version 0.29.0, it is now possible to drive humanoid models into the MMD runtime.

To be precise, HumanoidMmd provides the ability to create an MmdModel from a humanoid mesh

You can check out the test code here:
https://github.com/noname0310/babylon-mmd/blob/main/src/Test/Scene/humanoidMmdTestScene.ts
https://github.com/noname0310/babylon-mmd/blob/main/src/Test/Scene/humanoidMmdTestScene2.ts

Documentation will be written later, so if you have any questions, please open an issue