lokinko

lokinko's Stars

• ggerganov/llama.cpp

  LLM inference in C/C++

  Language:C++64.6k9.2k

• weishengying/tiny-flash-attention

  使用 cutlass 实现 flash-attention 精简版，具有教学意义

  Language:Cuda291

• LLMServe/DistServe

  Disaggregated serving system for Large Language Models (LLMs).

  Language:Jupyter Notebook27026

• liguodongiot/llm-action

  本项目旨在分享大模型相关技术原理以及实战经验。

  Language:HTML9k875

• NVlabs/DoRA

  [ICML2024 (Oral)] Official PyTorch implementation of DoRA: Weight-Decomposed Low-Rank Adaptation

  Language:Python55029

• microsoft/ToRA

  ToRA is a series of Tool-integrated Reasoning LLM Agents designed to solve challenging mathematical reasoning problems by interacting with tools [ICLR'24].

  Language:Python92671

• OpenBMB/MiniCPM-V

  MiniCPM-V 2.6: A GPT-4V Level MLLM for Single Image, Multi Image and Video on Your Phone

  Language:Python11.6k816

• BBuf/how-to-optim-algorithm-in-cuda

  how to optimize some algorithm in cuda.

  Language:Cuda1.4k116

• FasterDecoding/Medusa

  Medusa: Simple Framework for Accelerating LLM Generation with Multiple Decoding Heads

  Language:Jupyter Notebook2.2k147

• openai/CLIP

  CLIP (Contrastive Language-Image Pretraining), Predict the most relevant text snippet given an image

  Language:Jupyter Notebook24.6k3.2k

• facebookresearch/DiT

  Official PyTorch Implementation of "Scalable Diffusion Models with Transformers"

  Language:Python6k528

• NVIDIA/Megatron-LM

  Ongoing research training transformer models at scale

  Language:Python9.9k2.2k

• hyp1231/awesome-llm-powered-agent

  Awesome things about LLM-powered agents. Papers / Repos / Blogs / ...

  1.3k102

• galeselee/Awesome_LLM_System-PaperList

  Since the emergence of chatGPT in 2022, the acceleration of Large Language Model has become increasingly important. Here is a list of papers on accelerating LLMs, currently focusing mainly on inference acceleration, and related works will be gradually added in the future. Welcome contributions!

  1505

• datawhalechina/tiny-universe

  《大模型白盒子构建指南》：一个全手搓的Tiny-Universe

  Language:Python88887

• HqWu-HITCS/Awesome-Chinese-LLM

  整理开源的中文大语言模型，以规模较小、可私有化部署、训练成本较低的模型为主，包括底座模型，垂直领域微调及应用，数据集与教程等。

  14.7k1.4k

• TemporaryLoRA/Temp-LoRA

  Language:Python807

• deepseek-ai/DeepSeek-V2

  DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model

  3.4k127

• ModelTC/lightllm

  LightLLM is a Python-based LLM (Large Language Model) inference and serving framework, notable for its lightweight design, easy scalability, and high-speed performance.

  Language:Python2.3k187

• liguodongiot/llm-resource

  LLM全栈优质资源汇总

  Language:Shell30529

• DefTruth/Awesome-LLM-Inference

  📖A curated list of Awesome LLM Inference Paper with codes, TensorRT-LLM, vLLM, streaming-llm, AWQ, SmoothQuant, WINT8/4, Continuous Batching, FlashAttention, PagedAttention etc.

  2.4k159

• vllm-project/vllm

  A high-throughput and memory-efficient inference and serving engine for LLMs

  Language:Python26.1k3.8k

• wuhobin/blog-home

  一个干净简洁的个人作品集合主页

  Language:HTML22867

• systemdesign42/system-design

  A resource to help you pass system design interview and become good at work 👇

  11.9k1.2k

• rasbt/LLMs-from-scratch

  Implementing a ChatGPT-like LLM in PyTorch from scratch, step by step

  Language:Jupyter Notebook25.6k2.8k

• aishwaryanr/awesome-generative-ai-guide

  A one stop repository for generative AI research updates, interview resources, notebooks and much more!

  7.2k1.5k

• apple/pfl-research

  Simulation framework for accelerating research in Private Federated Learning

  Language:Jupyter Notebook28327

• comfyanonymous/ComfyUI

  The most powerful and modular diffusion model GUI, api and backend with a graph/nodes interface.

  Language:Python49.8k5.2k

• Eladlev/AutoPrompt

  A framework for prompt tuning using Intent-based Prompt Calibration

  Language:Python2k166

• google/gemma.cpp

  lightweight, standalone C++ inference engine for Google's Gemma models.

  Language:C++5.9k499