mllm

There are 128 repositories under mllm topic.

  • microsoft/unilm

    Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities

    Language:Python21.7k3091.4k2.7k

  • simular-ai/Agent-S

    Agent S: an open agentic framework that uses computers like a human

    Language:Python6.3k2622681

  • X-PLUG/MobileAgent

    Mobile-Agent: The Powerful GUI Agent Family

    Language:Python5.6k72167551

  • manycore-research/SpatialLM

    SpatialLM: Training Large Language Models for Structured Indoor Modeling

    Language:Python4k3331312

  • ant-research/MagicQuill

    [CVPR'25] Official Implementations for Paper - MagicQuill: An Intelligent Interactive Image Editing System

    Language:Python3.6k40121374

  • NExT-GPT/NExT-GPT

    Code and models for ICML 2024 paper, NExT-GPT: Any-to-Any Multimodal Large Language Model

    Language:Python3.6k61112360

  • atfortes/Awesome-LLM-Reasoning

    From Chain-of-Thought prompting to OpenAI o1 and DeepSeek-R1 🍓

    3.3k497194

  • SkyworkAI/Skywork-R1V

    Skywork-R1V is an advanced multimodal AI model series developed by Skywork AI (Kunlun Inc.), specializing in vision-language reasoning.

    Language:Python2.9k5413269

  • InternLM/InternLM-XComposer

    InternLM-XComposer2.5-OmniLive: A Comprehensive Multimodal System for Long-term Streaming Video and Audio Interactions

    Language:Python2.9k43425177

  • X-PLUG/mPLUG-DocOwl

    mPLUG-DocOwl: Modularized Multimodal Large Language Model for Document Understanding

    Language:Python2.2k33131130

  • cambrian-mllm/cambrian

    Cambrian-1 is a family of multimodal LLMs with a vision-centric design.

    Language:Python2k2183129

  • magic-research/Sa2VA

    🔥 Sa2VA: Marrying SAM2 with LLaVA for Dense Grounded Understanding of Images and Videos

    Language:Python1.2k235282

  • BAAI-DCAI/Bunny

    A family of lightweight multimodal models.

    Language:Python1k2013777

  • NVlabs/EAGLE

    Eagle: Frontier Vision-Language Models with Data-Centric Strategies

    Language:Python865313648

  • CircleRadon/Osprey

    [CVPR2024] The code for "Osprey: Pixel Understanding with Visual Instruction Tuning"

    Language:Python829134843

  • taco-group/OpenEMMA

    OpenEMMA, a permissively licensed open source "reproduction" of Waymo’s EMMA model.

    Language:Python79171896

  • BradyFU/Woodpecker

    ✨✨Woodpecker: Hallucination Correction for Multimodal Large Language Models

    Language:Python634161331

  • FoundationVision/Groma

    [ECCV2024] Grounded Multimodal Large Language Model with Localized Visual Tokenization

    Language:Python578264644

  • SkyworkAI/Vitron

    NeurIPS 2024 Paper: A Unified Pixel-level Vision LLM for Understanding, Generating, Segmenting, Editing

    Language:Python567152435

  • gokayfem/ComfyUI_VLM_nodes

    Custom ComfyUI nodes for Vision Language Models, Large Language Models, Image to Music, Text to Music, Consistent and Random Creative Prompt Generation

    Language:Python527712951

  • YingqingHe/Awesome-LLMs-meet-Multimodal-Generation

    🔥🔥🔥 A curated list of papers on LLMs-based multimodal generation (image, video, 3D and audio).

    Language:HTML50915530

  • Coobiw/MPP-LLaVA

    Personal Project: MPP-Qwen14B & MPP-Qwen-Next(Multimodal Pipeline Parallel based on Qwen-LM). Support [video/image/multi-image] {sft/conversations}. Don't let the poverty limit your imagination! Train your own 8B/14B LLaVA-training-like MLLM on RTX3090/4090 24GB.

    Language:Jupyter Notebook47263823

  • ZebangCheng/Emotion-LLaMA

    Emotion-LLaMA: Multimodal Emotion Recognition and Reasoning with Instruction Tuning

    Language:Python40389944

  • dvlab-research/LLMGA

    This project is the official implementation of 'LLMGA: Multimodal Large Language Model based Generation Assistant', ECCV2024 Oral

    Language:Python3948524

  • baaivision/EVE

    EVE Series: Encoder-Free Vision-Language Models from BAAI

    Language:Python34910209

  • Atomic-man007/Awesome_Multimodel_LLM

    Awesome_Multimodel is a curated GitHub repository that provides a comprehensive collection of resources for Multimodal Large Language Models (MLLM). It covers datasets, tuning techniques, in-context learning, visual reasoning, foundational models, and more. Stay updated with the latest advancement.

    3148221

  • X-PLUG/Youku-mPLUG

    Youku-mPLUG: A 10 Million Large-scale Chinese Video-Language Pre-training Dataset and Benchmarks

    Language:Python3016329

  • VITA-MLLM/Long-VITA

    ✨✨Long-VITA: Scaling Large Multi-modal Models to 1 Million Tokens with Leading Short-Context Accuracy

    Language:Python29115829

  • CircleRadon/TokenPacker

    The code for "TokenPacker: Efficient Visual Projector for Multimodal LLM", IJCV2025

    Language:Python2667238

  • DAMO-NLP-SG/VideoRefer

    [CVPR 2025] The code for "VideoRefer Suite: Advancing Spatial-Temporal Object Understanding with Video LLM"

    Language:Jupyter Notebook266869

  • bz-lab/AUITestAgent

    AUITestAgent is the first automatic, natural language-driven GUI testing tool for mobile apps, capable of fully automating the entire process of GUI interaction and function verification.

    25931322

  • X-PLUG/mPLUG-2

    mPLUG-2: A Modularized Multi-modal Foundation Model Across Text, Image and Video (ICML 2023)

    Language:Python22942520

  • TIGER-AI-Lab/Mantis

    Official code for Paper "Mantis: Multi-Image Instruction Tuning" [TMLR 2024]

    Language:Python22792420

  • IDEA-Research/ChatRex

    Code for ChatRex: Taming Multimodal LLM for Joint Perception and Understanding

    Language:Python2024157

  • zhipeixu/FakeShield

    🔥 [ICLR 2025] FakeShield: Explainable Image Forgery Detection and Localization via Multi-modal Large Language Models

    Language:Python19882717

  • Yui010206/SeViLA

    [NeurIPS 2023] Self-Chained Image-Language Model for Video Localization and Question Answering

    Language:Python18622721