/candle-vllm

Efficent platform for inference and serving local LLMs including an OpenAI compatible API server.

Primary LanguageRustMIT LicenseMIT

Watchers

No one’s watching this repository yet.