LizhenWangT/FaceVerse

When migrate to other standard arkit models, the effect is very bad

Opened this issue · 3 comments

youngstu commented

HI, Whether it complies with the standard arkit specification. When I retarget to other standard arkit models, the effect is very bad. Is there a mistake for the expression base order of faceverse_simple_v2.npy? Thanks.

youngstu commented

Have you verified other arkit-52 specification models, such as arkit and Metahuman 3d face model?

LizhenWangT commented

We test our method on metahuman using UE5. You need to:

  1. Comment out the eye-direction-related expressions, e.g. eyelookdown, eyelookin, etc.
  2. The mapping of pylinkface in UE5 is wrong, so you need to correct the mapping first.
  3. The defination of our blendshapes and metahuman seems not completely the same. The results are slightly worse but still workable.
youngstu commented
  1. I render with pytorch3d in the order of exp_ name_list and my arkit standard model are averaged and summed.
  2. The mouth effect is that the mouth is very shaking and the opening is small.
  3. And I tested several models of the arkit specification, and the results are the same.
  4. When I use iphonex live link to drive data, the effect is good.