FasterDecoding

Think deeper, decode faster

United States of America

Pinned Repositories

BitDelta
Language:Jupyter Notebook188 4 514
Medusa
Medusa: Simple Framework for Accelerating LLM Generation with Multiple Decoding Heads
Language:Jupyter Notebook2.4k 31 92163
REST
REST: Retrieval-Based Speculative Decoding, NAACL 2024
Language:C182 6 2111
SnapKV
Language:Python207 5 2210
TEAL
Language:Python102 3 101

FasterDecoding's Repositories

FasterDecoding/Medusa
Medusa: Simple Framework for Accelerating LLM Generation with Multiple Decoding Heads
Language:Jupyter Notebook2.4k 31 92163
FasterDecoding/SnapKV
Language:Python207 5 2210
FasterDecoding/BitDelta
Language:Jupyter Notebook188 4 514
FasterDecoding/REST
REST: Retrieval-Based Speculative Decoding, NAACL 2024
Language:C182 6 2111
FasterDecoding/TEAL
Language:Python102 3 101