- Patch partition
- Patch merging
- Relative position bias
- Feature map padding
- Self-attention in non-overlapped windows
- Shifted Window based Self-Attention
KimRass/Swin-Transformer
PyTorch implementation of 'Swin Transformer' (Liu et al., 2021) from scratch.
Python