About `window_size` in MOAT
Opened this issue · 3 comments
I found window_size
is None in MOAT
deeplab2/model/pixel_encoder/moat.py
Lines 347 to 360 in eb66c85
deeplab2/model/pixel_encoder/moat.py
Line 405 in eb66c85
Is only global attention used for segmentation tasks?
Thanks for your interest!
Please see
deeplab2/model/layers/moat_blocks.py
Line 329 in 7a01a71
Our setting can be found in the experimental sections on paper, but I can provide the information here:
For COCO object detection, we use window based attention for the third stage with size 14x14 and global attention for the fourth stage.
For ADE20K semantic segmentation, we use global attention for both third and fourth stages.
Thanks for your interest!
Please see
deeplab2/model/layers/moat_blocks.py
Line 329 in 7a01a71
for how to specify the desired window size for the use case.
Our setting can be found in the experimental sections on paper, but I can provide the information here: For COCO object detection, we use window based attention for the third stage with size 14x14 and global attention for the fourth stage. For ADE20K semantic segmentation, we use global attention for both third and fourth stages.
Thanks for your point out.
I also noticed the implementation of the global window
is flawed.
When using the global window size, the current implementation will still record a fixed window size, depending on the input size in the build stage. Therefore, if the given input size is different from the recorded size, the global
will be limited or direct raised error (e.g., smaller input size than recorded window size)
Thank you for finding this typo.
If you want to evaluate the model with an input size that is different from your training phase, you will need to create another model that is built with that input size and loads the weights. This is how the current tensorflow model works.