Is it very time-consuming to evaluate the model?
songbingze opened this issue · 3 comments
I tried to evaluate the model on a remote sensing image dataset and found that testing 1300 512 by 512 sized images would take approximately 7 hours. Is this normal?
It seems abnormal, but could you provide more context about your evaluation settings?
- Which model did you use? ViT-H, ViT-L, or ViT-B?
- Did you evaluate on CPU or GPU?
- How many objects are in each image? The speed is proportional to the total objects, not the images.
- Were there any other programs running on the machine during the evaluation?
According to my experience, it took ~30mins for the ViT-B model to evaluate SBD (2857 images, 6671 instances) on an NVIDIA RTX A6000 GPU.
Thank you so much for taking the time to reply to my message. I truly appreciate your help and expertise.
After reading your response, I think I understand why the speed is so slow. It seems that each of my images contains around 20-40 instances, and labeling them accurately requires labeling 11+ points, which can be quite challenging due to the small, scattered, and complex edges, as well as the blurred nature of remote sensing images. I understand that this can significantly affect the evaluation speed.
Once again, thank you for your valuable insights and assistance.
Yes, that makes a lot of sense. Thanks for the question!