/bsp

Primary LanguagePython

Batched Speculative Inference of LLM

Don't expect accuracy with fp16.