/hedgehog-porcupine-linear-attention

Code implementation in PyTorch of the paper "The Hedgehog & the Porcupine: Expressive Linear Attentions with Softmax Mimicry"

Primary LanguagePythonApache License 2.0Apache-2.0

hedgehog-porcupine-linear-attention

Code implementation in PyTorch of the paper "The Hedgehog & the Porcupine: Expressive Linear Attentions with Softmax Mimicry" at https://arxiv.org/abs/2402.04347