lyogavin/airllm

docker based or BareMetal serving

Opened this issue · 0 comments

dhandhalyabhavik commented

Wondering if any plans to implement to enable servings,

similar to vllm serving, it should support OpenAI compatible chat endpoints.