Align SMAL model to new 3D scan.
Opened this issue · 3 comments
Hi, may I ask for how to align SMAL model to a new 3D scan?
We already got the CT high-resolution scan. The code for the original SMAL model is not provided.
Therefore, I tried to fit the scan with the SMAL model based on Flame (https://github.com/Rubikplayer/flame-fitting). I follow the code of FLAME and use a similar objective function ( scan-to-mesh distance, landmarks distance, pose penalty, shape penalty with similar weights). I defined 54 points (from head to tail) on both the SMAL model and our CT scan using MeshLab as FLAME. However, the result of fitting SMAL to a scan based on FLAME code is very poor.
To eliminate the possibility that the SMAL model did not cover rodents, I also try to fit the Bear mesh provided in the result part of this SMALR project. It is also not working.
Could you please give me some advice on it? Is it possible to share the code about how to build a SMAL model based on scan?
Hi, I do not think SMAL can work for rodents, they are too different from the animals in the SMAL training set. FLAME is for faces, I guess it is not directly applicable to an articulated animal. I can imagine you also have poor results with the bear, as the point of doing the refinement with SMALR is that you cannot get a bear with SMAL, as it is also not in the training set. If you want to fit the model to a 3D scan you can use the code from Benjamin Biggs: https://github.com/benjiebob/SMALify. Best
…
On 28. May 2021, at 20:39, giles-hu @.***> wrote: Hi, may I ask for how to align SMAL model to a new 3D scan? We already got the CT high-resolution scan. The code for the original SMAL model is not provided. Therefore, I tried to fit the scan with the SMAL model based on Flame (https://github.com/Rubikplayer/flame-fitting https://github.com/Rubikplayer/flame-fitting). I follow the code of FLAME and use a similar objective function ( scan-to-mesh distance, landmarks distance, pose penalty, shape penalty with similar weights). I defined 54 points (from head to tail) on both the SMAL model and our CT scan using MeshLab as FLAME. However, the result of fitting SMAL to a scan based on FLAME code is very poor. To eliminate the possibility that the SMAL model did not cover rodents, I also try to fit the Bear mesh provided in the result part of this SMALR project. It is also not working. Could you please give me some advice on it? Is it possible to share the code about how to build a SMAL model based on scan? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#4>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABNYVFLJSPXPR3QXMOMRYFLTP7PONANCNFSM45XHEEYQ.
Hi,
Thanks a lot.
I've tried SMALify and added an MSEloss for landmarks in objective function. However, it still return poor result. I think it might be caused by that their objective function only contains a few regularization terms (in a very general way) and chamfer distance. In addition, it tries to fit all parameters at the same time.
I am now trying to using the GLoSS fitting on your SMAL paper (CVPR 2017). The objective function contains more regularization terms.
Could you please provide the information about the mesh segmentation of the SMAL model and some key matrix of the SMAL model like average pose displacements B_{s,i}, B_{p,i}?
Many thanks in advance.
Hope you have a good weekend.