shubham-goel/ucmr

Evaluation result cannot be reproduced

Closed this issue · 2 comments

xuxy09 commented

I followed the instructions to run the evaluation. However, I cannot reproduce the results in Table 2 of the paper. My output is mean iou = 0.637 mean qerr = 44.4 az_WS= 8.98 el_WS= 6.39 az_KL= inf el_KL= inf.

Could you please let me know how to reproduce the exact numbers?

shubham-goel commented

Hi @xuxy09. Thanks for asking. Your output seems correct. I suspect I might have run the camera pose training without setting a random seed while submitting camera pose results in the original paper. So it won't be possible to get the exact numbers in Table 2, but you'll get numbers that are almost the same as what we reported.

yangyouzi commented

Hello, No module named 'soft' appears when I try to run the project_ How to solve the error of renderer ''? Is the file missing?