cat538

cat538's Stars

• LargeWorldModel/LWM

  Language:Python7.1k6670547

• rizsotto/Bear

  Bear is a tool that generates a compilation database for clang tooling.

  Language:C++4.8k60427313

• InternLM/lmdeploy

  LMDeploy is a toolkit for compressing, deploying, and serving LLMs.

  Language:Python4.1k331.3k373

• htqin/awesome-model-quantization

  A list of papers, docs, codes about model quantization. This repo is aimed to provide the info for model quantization research, we are continuously improving the project. Welcome to PR the works (papers, repositories) that are missed by the repo.

  1.8k6112203

• HazyResearch/ThunderKittens

  Tile primitives for speedy kernels

  Language:Cuda1.5k252255

• gkamradt/LLMTest_NeedleInAHaystack

  Doing simple retrieval from LLM models at various context lengths to measure accuracy

  Language:Jupyter Notebook1.4k1525149

• NVIDIA/cccl

  CUDA Core Compute Libraries

  Language:C++1.1k311.3k127

• Vahe1994/AQLM

  Official Pytorch repository for Extreme Compression of Large Language Models via Additive Quantization https://arxiv.org/pdf/2401.06118.pdf and PV-Tuning: Beyond Straight-Through Estimation for Extreme LLM Compression https://arxiv.org/abs/2405.14852

  Language:Python1.1k2070171

• horseee/Awesome-Efficient-LLM

  A curated list for Efficient Large Language Models

  Language:Python1.1k42279

• HuangOwen/Awesome-LLM-Compression

  Awesome LLM compression research papers and tools.

  1k40363

• Cornell-RelaxML/quip-sharp

  Language:Python475116042

• NVIDIA/nvbench

  CUDA Kernel Benchmarking Library

  Language:Cuda472179663

• NVIDIA/cuCollections

  Language:C++4651718083

• mit-han-lab/qserve

  QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Serving

  Language:Python39192816

• microsoft/BitBLAS

  BitBLAS is a library to support mixed-precision matrix multiplications, especially for quantized LLM deployment.

  Language:Python322135329

• mirage-project/mirage

  A multi-level tensor algebra superoptimizer

  Language:C++310102118

• hahnyuan/LLM-Viewer

  Analyze the inference of Large Language Models (LLMs). Analyze aspects like computation, storage, transmission, and hardware roofline model in a user-friendly interface.

  Language:Python2652829

• Bruce-Lee-LY/cuda_hgemm

  Several optimization methods of half-precision general matrix multiplication (HGEMM) using tensor core with WMMA API and MMA PTX instruction.

  Language:Cuda25741261

• efeslab/Atom

  [MLSys'24] Atom: Low-bit Quantization for Efficient and Accurate LLM Serving

  Language:Cuda253111620

• KnowingNothing/MatmulTutorial

  A Easy-to-understand TensorOp Matmul Tutorial

  Language:C++2488927

• ModelTC/llmc

  This is the official PyTorch implementation of "LLMC: Benchmarking Large Language Model Quantization with a Versatile Compression Toolkit".

  Language:Python21391324

• jy-yuan/KIVI

  KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cache

  Language:Python20852418

• usyd-fsalab/fp6_llm

  An efficient GPU support for LLM inference with x-bit quantization (e.g. FP6,FP5).

  Language:Cuda1715914

• mit-han-lab/Quest

  [ICML 2024] Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inference

  Language:Cuda150376

• opengear-project/GEAR

  GEAR: An Efficient KV Cache Compression Recipefor Near-Lossless Generative Inference of LLM

  Language:Python13311811

• SqueezeBits/QUICK

  QUICK: Quantization-aware Interleaving and Conflict-free Kernel for efficient LLM inference

  Language:Python106166

• jeffreysijuntan/lloco

  The official repo for "LLoCo: Learning Long Contexts Offline"

  Language:Python104339

• mit-han-lab/lmquant

  Language:Python1002164

• naver-aics/lut-gemm

  Language:C++28214

• EfficientLLMSys/MuxServe

  Language:Jupyter Notebook10012