MediaPipe4U is an UnrealEngine plugin that integrated Google Mediapipe technology for capturing human motion (include body, fingers, face) and puppeteering 3D avatar through webcam, videos, and images in realtime.
| Unreal Engine | China Site | Global Site | Update |
|---|---|---|---|
| UE 5.0 | cowtransfer (China) | OneDrive | 2023-05-05 |
| UE 5.1 | cowtransfer (China) | OneDrive | 2023-04-23 |
Because the plugin is precompiled and contains a large number of C++ link symbols and debug symbols, it will cost 5G disk space after decompression (most files are UE-generated binaries in Intermediate).
Don't need to worry about disk usage, this is just disk usage during development, after the project is packaged, the plug-in disk usage is 200M only (most files are GStreamer dynamic library).
| Android Version | Download Link | Update |
|---|---|---|
| Android 7.0 or later | Download | 2023-04-21 |
About M4U Remoting
Note
This is a commercial license exclusive feature: capturing facial expressions from android device.
Free license only supports using in UE Editor, cannot be packaged this feature.
M4U Remoting Document
Demo Project
The demo project does not contain plugins, you need to download the plugin and copy content to the project's plugins folder to run.
Packaging Environment Requirements
- Visual Studio 2022
- WINSDK: 10.0.22621 or later (下载地址)
- Visual C++ Redistributable 14.35 or later (下载地址)
Video Tutorials (English)
Video Tutorials (Chinese)
If you have any questiongs, please check FAQ first. The problems listed there may be also yours. If you can’t find an answer in the FAQ, please post an issue. Private message or emal may cause the question to be mised .
Since the Windows version of MediaPipe does not support GPU inference, Windows relies on the CPU to inferring human pose estimation (see MediaPipe offical site for more details).
Hardware Evnrioment
CPU： AMD 3600 CPU
RAM: 32GB
GPU: Nvidia 1660s
Evaluation
Frame Rate： 18-24 fps
CPU usage：20% (Based on DEMO project)
Follow Me
[fix] UE5.0 材质打包版本不正确
[new] 🔥加入 M4U Remoting, 支持从安卓设备捕捉表情 （文档）。
[new] Android 面补工具开放下载（商业授权功能，免费授权无法打包）
[new] UE 编辑器工具新功能：
- LiveLink 追踪
- 位移、姿势校准
- 面容校准
[new] 统一位置、姿态校准，从 AnimationInstance 访问 CalibrationCountDownSeconds 等参数；现在，支持手动校准位置和姿势（:red_circle:Break Changes）。
[fix] 动补回调包发生错误时可能导致 texture 被占用无法释放
[fix] 退出编辑器时概率引发崩溃
[fix] UE 工具箱状态不同步