/Medusa

Medusa: Simple Framework for Accelerating LLM Generation with Multiple Decoding Heads

Primary LanguageJupyter NotebookApache License 2.0Apache-2.0

Watchers