/long-context-attention

USP: Unified (a.k.a. Hybrid, 2D) Sequence Parallel Attention for Long Context Transformers Model Training and Inference

Primary LanguagePythonApache License 2.0Apache-2.0

Watchers