Lightweight continous batching OpenAI compatibility using HuggingFace Transformers.
Primary LanguagePython