/sparse_attention

Examples of using sparse attention, as in "Generating Long Sequences with Sparse Transformers"

Primary LanguagePython