/memory_efficient_attention.pytorch

A human-readable PyTorch implementation of "Self-attention Does Not Need O(n^2) Memory" (Rabe&Staats'21).

Primary LanguagePythonApache License 2.0Apache-2.0

Stargazers