Performance when small batch size

[SongDoHou]

Hi, thank you for providing a awesome self-supervised learning research!

I'm wonder that how the performance will decrease when we use small batch size like between 128 ~ 512 for DEIT base model.

If we cannot use large batch size (ex: 1024) on base model, is it better to use smaller model with large batch size?

Thanks in advance!!