Advice on generating own tracking data

[Kartik-Teotia]

Hi,

First of all, amazing work!

I had questions about what are max_per and min_per doing in the code and how can we calulate them? And are expression coefficients the same as expression parameters obtained from a 3DMM model? I basically want to know how this combination is taking place and how can I generate my own data for it using a 3DMM tracker.

Thanks

[XuanGhahahaha]

Hi,

Thanks for your interest in our work.

max_per and min_per means element-wise maximum/minimum of our expression coefficients in train set. These variables are used to restrict the range of expression coefficients when inference, which could avoid the bad results when test expression is far from the train set distribution. They are also used for our expression-aware density grid update operation.

Because 3DMM model is a PCA model, which lacks semantic meaning for every basis, the expression coefficients here are not tracked with a 3DMM model, but with a blendshape model. In our experiments, we use the blendshape of FaceWarehouse for tracking.

[Kartik-Teotia]

Ah, is the blendshape coefficient-based tracking not available for public use?

[XuanGhahahaha]

It is not available here, that's because the tracking code is occupied by the Image Derivative Inc., which I unfortunately don't have right to make it public.

[Kartik-Teotia]

Do you think the method can work with PCA coefficients instead?

[XuanGhahahaha]

Hi, on theory, it should also work with PCA coefficients.

[cucdengjunli]

It is not available here, that's because the tracking code is occupied by the Image Derivative Inc., which I unfortunately don't have right to make it public.

I try to re produce it

[Ling27]

@cucdengjunli Have you been able to reproduce the tracking? Thank you!

[ZXisSpider]

@cucdengjunli Have you been able to reproduce the tracking? Thank you!

Have you tried using Deep3Dface as tracking to reproduce the results?