ROCm/vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

Easy, fast, and cheap LLM serving for everyone

| Documentation | Blog | Paper | Twitter/X | User Forum | Developer Slack |

---

Deprecation warning

Caution

The ROCm/vllm repository is retired, please use the upstream repository