kvcache.ai
KVCache.AI is a joint research project between MADSys and top industry collaborators, focusing on efficient LLM serving.
Pinned Repositories
custom_flashinfer
FlashInfer: Kernel Library for LLM Serving
ktransformers
A Flexible Framework for Experiencing Cutting-edge LLM Inference Optimizations
Mooncake
Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.
vllm
A high-throughput and memory-efficient inference and serving engine for LLMs
kvcache.ai's Repositories
kvcache-ai/Mooncake
Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.
kvcache-ai/ktransformers
A Flexible Framework for Experiencing Cutting-edge LLM Inference Optimizations
kvcache-ai/vllm
A high-throughput and memory-efficient inference and serving engine for LLMs
kvcache-ai/custom_flashinfer
FlashInfer: Kernel Library for LLM Serving