Can model predict gazing direction of eyeballs?
yilei0620 opened this issue · 2 comments
Excellent work for 3D face reconstruction from 2D images!
The showing gif and demo video also exhibit eyeballs gazing directions. However, from paper and experiments, I found that the model just outputs 3D mesh, FLAME pose parameters and the jaw rotation. Thus, can we get the rotation matrix of eyeballs? Or I guess that in the demo, the rotations of eyeballs are the same as the rotations of head? Is my guess right?
Yes one can potentially get the rotation of the eye balls. We kept the eye ball rotation fixed for our model for this project because we didn't use any 2D supervision for the eye ball tracking. So it would have been difficult to get the correct rotation matrix for the eye balls. Hence for now, the rotations of the eyeballs are the same as the rotation of the head as you said. But there one can potentially retrain our model with the eye balls tracking included if one has proper 2D supervision.
Got it! Thanks for your explaining!