Human Model is not working
imsamimalik opened this issue · 8 comments
Current Behavior
When I select the human option from the dropdown, it just keeps on loading.
This is what I am seeing:
What browser are you using?
Are there any errors in the console? (https://documentation.concretecms.org/tutorials/how-open-browser-console-view-errors)
My bad. There was some issue with Chrome Linux. It is working fine on Windows.
For The 3D model, have you looked into implementing something like Ready Player Me API? Since you are already using Mediapipe, it would be fairly easy to implement it using MediaPipe Holistic landmarks.
I am currently working on making a 3D character animated from a source video using MediaPipe. Let me know if I can be of any help.
I haven't looked at their API before, but I also don't see anything implying they have an API to rig using MediaPipe? Could you point me in the right direction? I'd love to include it of course if possible.
I got intrigued when I found this demo. He is extracting FaceLandmarks using MediaPipe and then using those landmarks he is Ready Player Me GLB model in the threejs.
I thought that if FaceLandmaks are possible then PoseLandmarks can be animated as well.
I also had a chance to look at Kalidokit. He is using VRM 3D models instead of Ready Player Me but the approach is the same (extracting full body landmarks using media pipe and then animating characters.)
For the 3D animation then you wouldn't need to send the whole file. At the Python backend, you can just extract the landmarks from the video and send those landmarks array as API response.
For the face demo, they are not using landmarks. Instead, they are using blendshapes.
Kalidokit is more promising to work directly from landmarks.
Sorry, forgot about the blendshapes. But, when the new Holistic API gets released(which should be in a few weeks, according to this), Kalidokit + MediaPipe Holistic + Ready Player Me seems like the meta for me.