nightflight-dk's Stars
microsoft/DeepSpeed
DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.
djsime1/awesome-flipperzero
🐬 A collection of awesome resources for the Flipper Zero device.
NVIDIA/open-gpu-kernel-modules
NVIDIA Linux open GPU kernel module source
triton-lang/triton
Development repository for the Triton language and compiler
NVIDIA/NeMo
A scalable generative AI framework built for researchers and developers working on Large Language Models, Multimodal, and Speech AI (Automatic Speech Recognition and Text-to-Speech)
huggingface/text-generation-inference
Large Language Model Text Generation Inference
triton-inference-server/server
The Triton Inference Server provides an optimized cloud and edge inferencing solution.
arcee-ai/mergekit
Tools for merging pretrained large language models.
Nukem9/dlssg-to-fsr3
Adds AMD FSR 3 Frame Generation to games by replacing Nvidia DLSS-G Frame Generation (nvngx_dlssg).
NVIDIA/NeMo-Guardrails
NeMo Guardrails is an open-source toolkit for easily adding programmable guardrails to LLM-based conversational systems.
turboderp/exllamav2
A fast inference library for running LLMs locally on modern consumer-class GPUs
mistralai/mistral-finetune
dusty-nv/jetson-containers
Machine Learning Containers for NVIDIA Jetson and JetPack-L4T
theotherp/nzbhydra2
Usenet meta search
tinygrad/open-gpu-kernel-modules
NVIDIA Linux open GPU with P2P support
triton-inference-server/tensorrtllm_backend
The Triton TensorRT-LLM Backend
triton-inference-server/client
Triton Python, C++ and Java client libraries, and GRPC-generated client examples for go, java and scala.
triton-inference-server/python_backend
Triton backend that enables pre-process, post-processing and other logic to be implemented in Python.
triton-inference-server/tutorials
This repository contains tutorials and examples for Triton Inference Server
SocialComplexityLab/life2vec
NVIDIA/TensorRT-Model-Optimizer
TensorRT Model Optimizer is a unified library of state-of-the-art model optimization techniques such as quantization and sparsity. It compresses deep learning models for downstream deployment frameworks like TensorRT-LLM or TensorRT to optimize inference speed on NVIDIA GPUs.
triton-inference-server/vllm_backend
HanGuo97/flute
Fast Matrix Multiplications for Lookup Table-Quantized LLMs
triton-inference-server/onnxruntime_backend
The Triton backend for the ONNX Runtime.
NVIDIA/trt-llm-as-openai-windows
This reference can be used with any existing OpenAI integrated apps to run with TRT-LLM inference locally on GeForce GPU on Windows instead of cloud.
NVIDIA-AI-IOT/tao-toolkit-triton-apps
Sample app code for deploying TAO Toolkit trained models to Triton
substratusai/vllm-docker
justinchuby/onnx-safetensors
Use safetensors with ONNX 🤗
pericles01/triton-automation
A way to automate the launch for triton server, prometheus and grafana using docker. The principal objective is the real-time monitoring for triton and AI metrics. Check the ReadMe for more!
tmckayus/triton-cloud-scripts
Some scripts to build a VM and launch the Triton Inference Server in the cloud