mhamilton723/STEGO

The DINO baseline is higher than STEGO.

xiao7199 opened this issue · 3 comments

xiao7199 commented

Thanks for sharing the code for your exciting work.

I have a question regarding the linear probing result of the DINO pipeline.

Since there are no details in the paper for evaluating the baseline, I write my own training script for linear probing and plug the learned classifier into the evaluation script provided in this repo for a fair comparison. From my experiment, DINO gets 42.73 mIOU, and STEGO gets 42.13 mIOU for COCO stuff, which means STEGO is not outperforming the DINO baseline for linear probing.
Can you provide more details on reproducing the baseline result for linear probing?
Thanks.

mhamilton723 commented

Its been a while since i ran ts baseline but i think i set projection_type to None and dropout to 0 and then trained and evaluated the system as usual. No projection means that no segmentation head is applied so its just DINO.

Let me know if this repros the numbers for you. If you indeed found that the pro code could be improved that is interesting as well. Thanks for your patience!

axkoenig commented

Hi there,
congrats on the great paper! We are publishing a follow-up study on STEGO in CVPR 23 Workshops, which also looks into the issue you describe. Table 2 might be interesting to you! :)
Cheers, Alex

  • 👍1
xiao7199 commented

Hi Alex,

Thanks for sharing the draft and the results. I think the results from Table 2 are consistent with what I've found.

Best, Xiao.

  • 👍1