/candle-vllm

Efficent platform for inference and serving local LLMs including an OpenAI compatible API server.

Primary LanguageRustMIT LicenseMIT

No issues in this repository yet.