/vattention

Dynamic Memory Management for Serving LLMs without PagedAttention

Primary LanguageCMIT LicenseMIT