jianyuheng

jianyuheng's Stars

• mlc-ai/mlc-llm

  Universal LLM Deployment Engine with ML Compilation

  Language:Python20.3k1801.5k1.7k

• Lightning-AI/litgpt

  20+ high-performance LLMs with recipes to pretrain, finetune and deploy at scale.

  Language:Python11.9k1148541.2k

• NVIDIA/TensorRT-LLM

  TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and build TensorRT engines that contain state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT-LLM also contains components to create Python and C++ runtimes that execute those TensorRT engines.

  Language:C++10k1132.3k1.3k

• axolotl-ai-cloud/axolotl

  Go ahead and axolotl questions

  Language:Python9k49754985

• microsoft/LMOps

  General technology for enabling AI capabilities w/ LLMs and MLLMs

  Language:Python3.9k57147302

• turboderp/exllamav2

  A fast inference library for running LLMs locally on modern consumer-class GPUs

  Language:Python3.8k35489290

• FasterDecoding/Medusa

  Medusa: Simple Framework for Accelerating LLM Generation with Multiple Decoding Heads

  Language:Jupyter Notebook2.5k3094174

• horseee/Awesome-Efficient-LLM

  A curated list for Efficient Large Language Models

  Language:Python1.6k425123

• Vahe1994/AQLM

  Official Pytorch repository for Extreme Compression of Large Language Models via Additive Quantization https://arxiv.org/pdf/2401.06118.pdf and PV-Tuning: Beyond Straight-Through Estimation for Extreme LLM Compression https://arxiv.org/abs/2405.14852

  Language:Python1.2k18106182

• OpenGVLab/OmniQuant

  [ICLR2024 spotlight] OmniQuant is a simple and powerful quantization technique for LLMs.

  Language:Python790178861

• mobiusml/hqq

  Official implementation of Half-Quadratic Quantization (HQQ)

  Language:Python7721612379

• SqueezeAILab/SqueezeLLM

  [ICML 2024] SqueezeLLM: Dense-and-Sparse Quantization

  Language:Python681182844

• princeton-nlp/LLM-Shearing

  [ICLR 2024] Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruning

  Language:Python598237351

• segmind/distill-sd

  Segmind Distilled diffusion

  Language:Python595171738

• mit-han-lab/qserve

  QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Serving

  Language:Python49684131

• hao-ai-lab/Consistency_LLM

  [ICML 2024] CLLMs: Consistency Large Language Models

  Language:Python38881117

• spcl/QuaRot

  Code for Neurips24 paper: QuaRot, an end-to-end 4-bit inference of large language models.

  Language:Python366106235

• Cornell-RelaxML/QuIP

  Code for paper: "QuIP: 2-Bit Quantization of Large Language Models With Guarantees"

  Language:Python36191533

• Nota-NetsPresso/BK-SDM

  A Compressed Stable Diffusion for Efficient Text-to-Image Generation [ECCV'24]

  Language:Python285104319

• OpenGVLab/EfficientQAT

  EfficientQAT: Efficient Quantization-Aware Training for Large Language Models

  Language:Python26142619

• hemingkx/Spec-Bench

  Spec-Bench: A Comprehensive Benchmark and Unified Evaluation Platform for Speculative Decoding (ACL 2024 Findings)

  Language:Python24431730

• MFaceTech/InstantID

  Language:Python150111528

• taprosoft/llm_finetuning

  Convenient wrapper for fine-tuning and inference of Large Language Models (LLMs) with several quantization techniques (GTPQ, bitsandbytes)

  Language:Python1473623

• jaymody/speculative-sampling

  Simple implementation of Speculative Sampling in NumPy for GPT-2.

  Language:Python922810

• ruiqixu37/distill_diffusion

  Implementation of the 2023 CVPR Award Candidate: On Distillation of Guided Diffusion Models

  Language:Python48543

• LiqunMa/FBI-LLM

  FBI-LLM: Scaling Up Fully Binarized LLMs from Scratch via Autoregressive Distillation

  Language:Python47412

• Qualcomm-AI-research/lr-qat

  Language:Python36332

• Skhaki18/optin-transformer-pruning

  [ICLR 2024] The Need for Speed: Pruning Transformers with One Recipe

  Language:Python26452

• MFaceTech/AIGC-SD-Acceleration

  Language:Python25313

• onliwad101/FlexRound_LRQ

  FlexRound (ICML 2023) & LRQ

  Language:Python10