Error when evaluating by running “bash eval/evaluate.sh configs/train_pix3d.yaml"
Closed this issue · 3 comments
Hi,
I am trying to run "bash eval/evaluate.sh configs/train_pix3d.yaml" but got the following error:
Is there any package or external directory missing in the code? Looking forward to your kind reply. Thank you very much!
Hi! Sorry, I forgot to upload the external folder, I have already updated the code and README, please update your local files and build the external evaluation libs first. Thank you very much!
Hi thank you very much for your reply! I run the evaluation.sh successfully! However, I retrained the model and got very different results compared to the one in the paper (Pix3D dataset):
Btw, I trained on 4x4090 gpu so I reduced the batch size to 32, but the result should not differ very much? Did I understand/use the evaluation code incorrectly? How to get the F-score / CD shown in the paper?
It looks like the f-score result is a bit strange. I just used the latest code to evaluate the Pix3d dataset again but this problem did not occur.
Please check whether you have updated the eval code, because when uploading the external folder, I organized the eval code according to the InstPIFu method. Some code in the external folder has been moved to evaluate_front3d.py following InstPIFu.
If the problem is still not solved, please email me your evaluation code, although I think using the latest eval codes, there should be no problem.
Regarding your other question, reducing the batch size will have a bad impact on the results, because a larger batch size will make the gradient more stable, which is a bit important in the reconstruction task, but I think the results you reproduced are mainly The f-score looks strange. Please follow the above method and then contact me.