NVlabs/RADIO

Worth updating default norm arg of intermediate layer features to be consistent with DINOv2?

Opened this issue · 0 comments

Nik-V9 commented

Hi, I just wanted to flag this in case you want to make it consistent with DINOv2's encoder.

I see that based on timm, you set the default flag for the norm of intermediate layers to False here:

RADIO/radio/radio_model.py

Line 172 in 41ec552

norm: bool = False,

However, DINOv2's intermediate feature return function uses norm=True as default param: https://github.com/facebookresearch/dinov2/blob/e1277af2ba9496fbadf7aec6eba56e8d882d1e35/dinov2/models/vision_transformer.py#L304

Feel free to close this issue if you think the intended behavior is to use unnormalized intermediate layer features.