Rotation Estimation Mismatch

[RahulJain7]

I am trying to estimate the rotation matrix using a car model. I have tried adjusting various parameters in the config section example the camera matrix, resolution, radius. The problem I am facing is that the model predicts certain crops perfectly whereas for some crops the predictions are far from matching. Following are the results
Image Crop
Prediction

Image Crop
Prediction

Image crop based on cad model (from Meshlab)
Prediction

As you can see the rotation estimate for the first crop is completely wrong where as the second one is almost perfect. Also, the image crops of any pose of the cad model (the third sample above) seems to work perfectly. Likewise I have tried with many crops of real images but most of the predictions are far off. I have tried with various camera intrinsics and resolution but a lot of crops are predicted completely wrong. As for training i have taken all the default input like number of images iterations and training datasets. Is it because the real model has different textures at different parts unlike the cad model which is texture less (because crops of 3d model are working perfectly)? If yes, is there any way where i can increase the model accuracy ?

[MartinSmeyer]

Hi @RahulJain7,

Is it because the real model has different textures at different parts unlike the cad model which is texture less (because crops of 3d model are working perfectly)? If yes, is there any way where i can increase the model accuracy ?

Yes that is the reason, it is an RGB method so you can't expect it to work well without the texture information if the object itself has texture. You can model the vertex colors for your specific car using Blender or some other software.

Here are some tutorials for Blender:
https://www.youtube.com/watch?v=8mNk6r_bwxI
https://www.youtube.com/watch?v=M9sgbxP6i78