wrong initialization of pose

[junhahyung]

Again, thank you for your great work :)

in preprocessing code, global 3dim pose is initialized as zero and concatenated at the front of 6dim pose from DECA.
However, 6dim DECA pose consists of 3dim global pose + 3dim jaw pose,
so my understanding is that the code should be

pose = torch.cat([pose[:,:3], torch.zeros_like(pose[:, :3]), pose[:,3:]], dim=1)

because you should zero initialize the neck pose(which is not given by the deca model) to zero,
and the final 15dim full pose should be in order of [global, neck, jaw, eye1, eye2]

Maybe I misunderstood something, so would be great if you check this issue.

Thanks :)

[zhengyuf]

Hi,

Global pose and neck pose are rather ambiguous in face trackers. We know that in our data capture, the global pose is close to zero since the upper body doesn't move much, therefore, we use the DECA global pose as our neck pose, and further optimize the poses with 2D landmark energy.
If we set neck pose to zero, IMavatar will transform the neck and upper body with the head, which might lead to problems.

Yufeng