moe

There are 181 repositories under moe topic.

  • LLaMA-Factory

    hiyouga/LLaMA-Factory

    Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024)

    Language:Python58.4k2917.4k7.2k

  • sgl-project/sglang

    SGLang is a fast serving framework for large language models and vision language models.

    Language:Python17.9k1173.2k2.9k

  • NVIDIA/TensorRT-LLM

    TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and support state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT-LLM also contains components to create Python and C++ runtimes that orchestrate the inference execution in performant way.

    Language:C++11.6k1202.9k1.7k
  • Bangumi

    czy0729/Bangumi

    :electron: An unofficial https://bgm.tv ui first app client for Android and iOS, built with React Native. 一个无广告、以爱好为驱动、不以盈利为目的、专门做 ACG 的类似豆瓣的追番记录，bgm.tv 第三方客户端。为移动端重新设计，内置大量加强的网页端难以实现的功能，且提供了相当的自定义选项。 目前已适配 iOS / Android。

    Language:TypeScript4.8k26265153

  • flashinfer-ai/flashinfer

    FlashInfer: Kernel Library for LLM Serving

    Language:Cuda3.7k30255496

  • zai-org/GLM-4.5

    GLM-4.5: An open-source large language model designed for intelligent agents by Z.ai

    Language:Python2.6k259

  • PKU-YuanGroup/MoE-LLaVA

    【TMM 2025🔥】 Mixture-of-Experts for Large Vision-Language Models

    Language:Python2.2k2397141

  • MoonshotAI/MoBA

    MoBA: Mixture of Block Attention for Long-Context LLMs

    Language:Python1.9k2524115

  • davidmrau/mixture-of-experts

    PyTorch Re-Implementation of "The Sparsely-Gated Mixture-of-Experts Layer" by Noam Shazeer et al. https://arxiv.org/abs/1701.06538

    Language:Python1.2k726110

  • pjlab-sys4nlp/llama-moe

    ⛷️ LLaMA-MoE: Building Mixture-of-Experts from LLaMA with Continual Pre-training (EMNLP 2024)

    Language:Python98782461

  • microsoft/Tutel

    Tutel MoE: Optimized Mixture-of-Experts Library, Support GptOss/DeepSeek/Kimi-K2/Qwen3 using FP8/NVFP4/MXFP4

    Language:C9131772103

  • sail-sg/Adan

    Adan: Adaptive Nesterov Momentum Algorithm for Faster Optimizing Deep Models

    Language:Python79773569

  • open-compass/MixtralKit

    A toolkit for inference and evaluation of 'mixtral-8x7b-32kseqlen' from Mistral AI

    Language:Python76981677

  • ScienceOne-AI/DeepSeek-671B-SFT-Guide

    An open-source solution for full parameter fine-tuning of DeepSeek-V3/R1 671B, including complete code and scripts from training to inference, as well as some practical experiences and conclusions. (DeepSeek-V3/R1 满血版 671B 全参数微调的开源解决方案，包含从训练到推理的完整代码和脚本，以及实践中积累一些经验和结论。)

    Language:Python75671694

  • ymcui/Chinese-Mixtral

    中文Mixtral混合专家大模型（Chinese Mixtral MoE LLMs）

    Language:Python608151144

  • mindspore-courses/step_into_llm

    MindSpore online courses: Step into LLM

    Language:Jupyter Notebook477841122

  • kokororin/pixiv.moe

    😘 A pinterest-style layout site, shows illusts on pixiv.net order by popularity.

    Language:TypeScript365114942

  • weigao266/Awesome-Efficient-Arch

    Speed Always Wins: A Survey on Efficient Architectures for Large Language Models

    322

  • LISTEN-moe/android-app

    Official LISTEN.moe Android app

    Language:Kotlin27585725

  • SkyworkAI/MoH

    MoH: Multi-Head Attention as Mixture-of-Head Attention

    Language:Python2723314

  • inferflow/inferflow

    Inferflow is an efficient and highly configurable inference engine for large language models (LLMs).

    Language:C++24881725

  • SkyworkAI/MoE-plus-plus

    [ICLR 2025] MoE++: Accelerating Mixture-of-Experts Methods with Zero-Computation Experts

    Language:Python2392111

  • libgdx/gdx-pay

    A libGDX cross-platform API for InApp purchasing.

    Language:Java2323518888

  • IBM/ModuleFormer

    ModuleFormer is a MoE-based architecture that includes two different types of experts: stick-breaking attention heads and feedforward experts. We released a collection of ModuleFormer-based Language Models (MoLM) ranging in scale from 4 billion to 8 billion parameters.

    Language:Python2259611

  • inclusionAI/Ling

    Ling is a MoE LLM provided and open-sourced by InclusionAI.

    Language:Python1992012

  • shufangxun/LLaVA-MoD

    [ICLR 2025] LLaVA-MoD: Making LLaVA Tiny via MoE-Knowledge Distillation

    Language:Python1959107

  • cocowy1/SMoE-Stereo

    [ICCV 2025 Highlight] 🌟🌟🌟 Learning Robust Stereo Matching in the Wild with Selective Mixture-of-Experts

    Language:Python172

  • LINs-lab/DynMoE

    [ICLR 2025] Dynamic Mixture of Experts: An Auto-Tuning Approach for Efficient Transformer Models

    Language:Python12911410

  • Facico/GOAT-PEFT

    [ICML2025] Make LoRA Great Again: Boosting LoRA with Adaptive Singular Values and Mixture-of-Experts Optimization Alignment

    Language:Python12612

  • junchenzhi/Awesome-LLM-Ensemble

    A curated list of Awesome-LLM-Ensemble papers for the survey "Harnessing Multiple Large Language Models: A Survey on LLM Ensemble"

    1191011

  • kyegomez/SwitchTransformers

    Implementation of Switch Transformers from the paper: "Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity"

    Language:Python1192312

  • shalldie/chuncai

    A lovely Page Wizard, is responsible for selling moe.

    Language:TypeScript1138334

  • kyegomez/MoE-Mamba

    Implementation of MoE Mamba from the paper: "MoE-Mamba: Efficient Selective State Space Models with Mixture of Experts" in Pytorch and Zeta

    Language:Python110547

  • LISTEN-moe/desktop-app

    Official LISTEN.moe Desktop Client

    Language:Vue1033229

  • Simplifine-gamedev/Simplifine

    🚀 Easy, open-source LLM finetuning with one-line commands, seamless cloud integration, and popular optimization frameworks. ✨

    Language:Python94214

  • marisukukise/japReader

    japReader is an app for breaking down Japanese sentences and tracking vocabulary progress

    Language:JavaScript904126