/worker-vllm

The RunPod worker template for serving our large language model endpoints. Powered by vLLM.

Primary LanguagePythonMIT LicenseMIT

Watchers