fake0fan's Stars
meta-llama/llama
Inference code for Llama models
hpcaitech/ColossalAI
Making large AI models cheaper, faster and more accessible
microsoft/DeepSpeed
DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.
docker/compose
Define and run multi-container applications with Docker
google/jax
Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more
vllm-project/vllm
A high-throughput and memory-efficient inference and serving engine for LLMs
sunface/rust-course
“连续六年成为全世界最受喜爱的语言，无 GC 也无需手动内存管理、极高的性能和安全性、过程/OO/函数式编程、优秀的包管理、JS 未来基石" — 工作之余的第二语言来试试 Rust 吧。<<Rust语言圣经>>拥有全面且深入的讲解、生动贴切的示例、德芙般丝滑的内容，甚至还有JS程序员关注的 WASM 和 Deno 等专题。这可能是目前最用心的 Rust 中文学习教程 / Book
triton-lang/triton
Development repository for the Triton language and compiler
ggerganov/ggml
Tensor library for machine learning
NVIDIA/Megatron-LM
Ongoing research training transformer models at scale
NVIDIA/TensorRT-LLM
TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and build TensorRT engines that contain state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT-LLM also contains components to create Python and C++ runtimes that execute those TensorRT engines.
NVIDIA/cuda-samples
Samples for CUDA Developers which demonstrates features in CUDA Toolkit
NVIDIA/FasterTransformer
Transformer related optimization, including BERT, GPT
NVIDIA/cutlass
CUDA Templates for Linear Algebra Subroutines
facebookincubator/AITemplate
AITemplate is a Python framework which renders neural network into high performance CUDA/HIP C++ code. Specialized for FP16 TensorCore (NVIDIA GPU) and MatrixCore (AMD GPU) inference.
stellargraph/stellargraph
StellarGraph - Machine Learning on Graphs
pykeen/pykeen
🤖 A Python library for learning and evaluating knowledge graph embeddings
openai/blocksparse
Efficient GPU kernels for block-sparse matrix multiplication and convolution
kvcache-ai/Mooncake
Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.
zhuzilin/ring-flash-attention
Ring attention implementation with flash attention
quiver-team/torch-quiver
PyTorch Library for Low-Latency, High-Throughput Graph Learning on GPUs.
feifeibear/long-context-attention
USP: Unified (a.k.a. Hybrid, 2D) Sequence Parallel Attention for Long Context Transformers Model Training and Inference
PKU-DAIR/Hetu
A high-performance distributed deep learning system targeting large-scale and automated distributed training.
microsoft/mscclpp
MSCCL++: A GPU-driven communication stack for scalable AI applications
RUCAIBox/Negative-Sampling-Paper
This repository collects 100 papers related to negative sampling methods.
galeselee/Awesome_LLM_System-PaperList
Since the emergence of chatGPT in 2022, the acceleration of Large Language Model has become increasingly important. Here is a list of papers on accelerating LLMs, currently focusing mainly on inference acceleration, and related works will be gradually added in the future. Welcome contributions!
iDC-NEU/NeutronStarLite
A Distributed GNN system
wzsh/wmma_tensorcore_sample
Matrix Multiply-Accumulate with CUDA and WMMA( Tensor Core)
ceruleangu/Block-Sparse-Benchmark
Benchmark for matrix multiplications between dense and block sparse (BSR) matrix in TVM, blocksparse (Gray et al.) and cuSparse.
uma-pi1/dist-kge