Hydragen: High-Throughput LLM Inference with Shared Prefixes
Primary LanguagePythonApache License 2.0Apache-2.0