triton-inference-server
There are 101 repositories under triton-inference-server topic.
NVIDIA/GenerativeAIExamples
Generative AI reference workflows optimized for accelerated infrastructure and microservice architecture.
CoinCheung/BiSeNet
Add bisenetv2. My implementation of BiSeNet
isarsoft/yolov4-triton-tensorrt
This repository deploys YOLOv4 as an optimized TensorRT engine to Triton Inference Server
npuichigo/openai_trtllm
OpenAI compatible API for TensorRT LLM triton backend
NetEase-Media/grps
Deep Learning Deployment Framework: Supports tf/torch/trt/trtllm/vllm and other NN frameworks. Support dynamic batching, and streaming modes. It is dual-language compatible with Python and C++, offering scalability, extensibility, and high performance. It helps users quickly deploy models and provide services through HTTP/RPC interfaces.
torchpipe/torchpipe
Serving Inside Pytorch
triton-inference-server/onnxruntime_backend
The Triton backend for the ONNX Runtime.
clearml/clearml-serving
ClearML - Model-Serving Orchestration and Repository Solution
kamalkraj/stable-diffusion-tritonserver
Deploy stable diffusion model with onnx/tenorrt + tritonserver
NVIDIA-ISAAC-ROS/isaac_ros_dnn_inference
NVIDIA-accelerated DNN model inference ROS 2 packages using NVIDIA Triton/TensorRT for both Jetson and x86_64 with CUDA-capable GPU
notAI-tech/fastDeploy
Deploy DL/ ML inference pipelines with minimal extra code.
Koldim2001/TrafficAnalyzer
Анализ трафика на круговом движении с использованием компьютерного зрения
akiragy/recsys_pipeline
Build Recommender System with PyTorch + Redis + Elasticsearch + Feast + Triton + Flask. Vector Recall, DeepFM Ranking and Web Application.
trinhtuanvubk/Diff-VC
Diffusion Model for Voice Conversion
bug-developer021/YOLOV5_optimization_on_triton
Compare multiple optimization methods on triton to imporve model service performance
rtzr/tritony
Tiny configuration for Triton Inference Server
chiehpower/Setup-deeplearning-tools
Set up CI in DL/ cuda/ cudnn/ TensorRT/ onnx2trt/ onnxruntime/ onnxsim/ Pytorch/ Triton-Inference-Server/ Bazel/ Tesseract/ PaddleOCR/ NVIDIA-docker/ minIO/ Supervisord on AGX or PC from scratch.
omarabid59/yolov8-triton
Provides an ensemble model to deploy a YoloV8 ONNX model to Triton
k9ele7en/Triton-TensorRT-Inference-CRAFT-pytorch
Advanced inference pipeline using NVIDIA Triton Inference Server for CRAFT Text detection (Pytorch), included converter from Pytorch -> ONNX -> TensorRT, Inference pipelines (TensorRT, Triton server - multi-format). Supported model format for Triton inference: TensorRT engine, Torchscript, ONNX
Bobo-y/triton_ensemble_model_demo
triton server ensemble model demo
light-hat/smart_ids
🧠🛡️ Web service for detecting network attacks in PCAP using ML.
openhackathons-org/End-to-End-Computer-Vision
This repository is an AI bootcamp material that consist of a workflow for computer vision
olibartfast/computer-vision-triton-cpp-client
C++ application to perform computer vision tasks using Nvidia Triton Server for model inference
Curt-Park/serving-codegen-gptj-triton
Serving Example of CodeGen-350M-Mono-GPTJ on Triton Inference Server with Docker and Kubernetes
inferless/triton-co-pilot
Generate Glue Code in seconds to simplify your Nvidia Triton Inference Server Deployments
tonhathuy/tensorrt-triton-magface
Magface Triton Inferece Server Using Tensorrt
Biano-AI/serving-compare-middleware
FastAPI middleware for comparing different ML model serving approaches
redis-applied-ai/redis-feast-gcp
A demo of Redis Enterprise as the Online Feature Store deployed on GCP with Feast and NVIDIA Triton Inference Server.
YeonwooSung/MLOps
Miscellaneous codes and writings for MLOps
detail-novelist/novelist-triton-server
Deploy KoGPT with Triton Inference Server
dpressel/reserve
FastAPI + WebSockets + SSE service to interface with Triton/Riva ASR
duydvu/triton-inference-server-web-ui
Triton Inference Server Web UI
levipereira/deepstream-yolo-triton-server-rtsp-out
The Purpose of this repository is to create a DeepStream/Triton-Server sample application that utilizes yolov7, yolov7-qat, yolov9 models to perform inference on video files or RTSP streams.
tech4ai/t4ai-signature-detect-server
This project provides a pipeline for deploying and performing inference with the YOLOv8 object detection model using the Triton Inference Server. It supports integration with local systems, Docker-based setups, or Google Cloud’s Vertex AI. The repository includes scripts for automated deployment, benchmarks and GUI inference.
ybai789/yolov8-triton-tensorrt
Provides an ensemble model to deploy a YOLOv8 TensorRT model to Triton
yas-sim/openvino-model-server-wrapper
Python wrapper class for OpenVINO Model Server. User can submit inference request to OVMS with just a few lines of code.