Pinned Repositories
FourierTransformer
The official Pytorch implementation of the paper "Fourier Transformer: Fast Long Range Modeling by Removing Sequence Redundancy with FFT Operator" (ACL 2023 Findings)
attention_sinks
Extend existing LLMs way beyond the original training length with constant memory usage, without retraining
Fovea-Transformer