/MagicDec

Breaking Throughput-Latency Trade-off for Long Sequences with Speculative Decoding

Primary LanguageJavaScriptApache License 2.0Apache-2.0

Issues