duanzhaol's Stars
AlibabaPAI/FLASHNN
predibase/lorax
Multi-LoRA inference server that scales to 1000s of fine-tuned LLMs
anti-entropy123/show-me-your-plan
bytedance/flux
A fast communication-overlapping library for tensor parallelism on GPUs.
AlibabaPAI/llumnix
Efficient and easy multi-instance LLM serving
microsoft/vattention
Dynamic Memory Management for Serving LLMs without PagedAttention
kvcache-ai/Mooncake
Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.
tyler-griggs/melange-release
wxdwfc/reproduce-gpt
A repo to record the followup of taking the course of reproducing GPT2
zhpmatrix/PaperReading
每天阅读过的论文的简要笔记
xdit-project/xDiT
xDiT: A Scalable Inference Engine for Diffusion Transformers (DiTs) on multi-GPU Clusters
HigherOrderCO/Bend
A massively parallel, high-level programming language
naklecha/llama3-from-scratch
llama3 implementation one matrix multiplication at a time
wdndev/llm_interview_note
主要记录大语言大模型(LLMs) 算法(应用)工程师相关的知识及面试题
StarryVae/RDMA-tutorial
aceliuchanghong/FAQ_Of_LLM_Interview
大模型算法岗面试题(含答案):常见问题和概念解析 "大模型面试题"、"算法岗面试"、"面试常见问题"、"大模型算法面试"、"大模型应用基础"
YangQianli92/LLMs_interview_notes
LLMs interview notes and answers:该仓库主要记录大模型(LLMs)算法工程师相关的面试题和参考答案
LLMServe/DistServe
Disaggregated serving system for Large Language Models (LLMs).
James-QiuHaoran/LLM-serving-with-proxy-models
Efficient Interactive LLM Serving with Proxy Model-based Sequence Length Prediction
guoweiu/Fuyao
Source code of Fuyao, built on Nightcore
HPMLL/BurstGPT
A ChatGPT(GPT-3.5) & GPT-4 Workload Trace to Optimize LLM Serving Systems
km1994/LLMsNineStoryDemonTower
【LLMs九层妖塔】分享 LLMs在自然语言处理(ChatGLM、Chinese-LLaMA-Alpaca、小羊驼 Vicuna、LLaMA、GPT4ALL等)、信息检索(langchain)、语言合成、语言识别、多模态等领域(Stable Diffusion、MiniGPT-4、VisualGLM-6B、Ziya-Visual等)等 实战与经验。
chenhongyu2048/LLM-inference-optimization-paper
Summary of some awesome work for optimizing LLM inference
sangyc10/CUDA-code
zeux/calm
CUDA/Metal accelerated language model inference
chenzomi12/AISystem
AISystem 主要是指AI系统,包括AI芯片、AI编译器、AI推理和训练框架等AI全栈底层技术
chenzomi12/DeepLearningSystem
AI Infra主要是指AI的基础建设,包括AI芯片、AI编译器、AI推理和训练框架等AI全栈底层技术。
galeselee/Awesome_LLM_System-PaperList
Since the emergence of chatGPT in 2022, the acceleration of Large Language Model has become increasingly important. Here is a list of papers on accelerating LLMs, currently focusing mainly on inference acceleration, and related works will be gradually added in the future. Welcome contributions!
DefTruth/CUDA-Learn-Notes
🎉CUDA/C++ 笔记 / 技术博客: fp32、fp16/bf16、fp8/int8、flash_attn、sgemm、sgemv、warp/block reduce、dot prod、elementwise、softmax、layernorm、rmsnorm、hist etc.
ZuodaoTech/everyone-can-use-english
人人都能用英语