uzh-rpg/RVT

zoom in augmentation

Closed this issue · 5 comments

Peterande commented

Hello, may I ask why zoom in cannot be used on streaming data?

magehrig commented

Hi @Peterande
In principle, it would be possible (you would have to change the implementation). However, the issue with zoom in augmentation in streaming mode is that you might end up zooming into a window that contains very little or even no labels for the whole sequence (e.g. middle-top part of the FoV has very few labels and might be a bad spot to zoom in).
In the random sampling mode, we check that we have at least 1 full label in the sequence per batch sample to avoid this issue to some degree.

Peterande commented

@magehrig Thank you for your reply. Have you verified that adding zoom-in will affect the performance?

magehrig commented

Yes it's in the ablation study (table 7) in the paper. Removing zoom-in negatively affects performance

Peterande commented

Allow me to correct my question. Have you verified that keeping zoom-in in streaming mode negatively affects performance?

magehrig commented

No. I have never attempted to make it work in streaming mode.