text-generation-inference
There are 9 repositories under text-generation-inference topic.
huggingface/optimum-benchmark
🏋️ A unified multi-backend utility for benchmarking Transformers, Timm, PEFT, Diffusers and Sentence-Transformers with full support of Optimum's hardware optimizations & quantization schemes.
InftyAI/llmaz
☸️ Easy, advanced inference platform for large language models on Kubernetes. 🌟 Star to support our work!
aws-samples/amazon-sagemaker-llama2-response-streaming-recipes
Amazon SageMaker Llama 2 Inference via Response Streaming
magichub-opensource/CLAM-Conversational-Language-AI-from-MagicData
This repo introduces MagicData-CLAM, a Chinese SFT dataset, and provides to the community two relevant models that we finetuned. Contact business@magicdatatech.com for more information.
yjg30737/pyqt-text-generation-inference-gui
GUI version of text-generation-inference
aisingapore/sealion-tgi
Serve the AI Singapore SEA-LION model ⚛ with TGI
dcalaprice/modal-sqlcoder
Deploy the Defog sqlcoder2 llm on Modal (https://modal.com) using Hugging Face Text Generation Inference (TGI)
HyperBlaze456/risu-backend-python
RisuAI backend with python only. TextGen works, need more memory related updates
yjg30737/windows-text-generation-inference-example
Text Generation Interference example in Windows (docker, WSL is needed)