/LLaMA-Cult-and-More

Keeping Track of Affordable LLMs, 🦙 Cult and More

MIT LicenseMIT

This repository is dedicated to tracking affordable but powerful language models (LLMs).
Filtered and reformatted version in chart.md.

survey-gif-test

Keeping track of affordable LLMs

  • 05/04: TIP(Dual Text-Image Prompting) a DALLE2/StableDiffusion-2 enhanced LLM that can generate coherent and authentic multimodal procedural plans toward a high-level goal (tweet)
  • 05/04: GPTutor, a ChatGPT-powered tool for code explanation (tweet)
  • 05/04: Chatbot Arena: Benchmarking LLMs in the Wild with Elo Ratings (blog, tweet)
  • 05/03: Modular/Mojo, a new Python-compatible language with a parallelizing compiler that can import Python libraries, combines the usability of Python with the performance of C, unlocking unparalleled programmability of AI hardware and extensibility of AI models. Only Limited notebook released for now. (tweet, blog, doc)
  • 05/01: "Are Emergent Abilities of Large Language Models a Mirage?" alternative explanation for emergent abilities, strong supporting evidence that emergent abilities may not be a fundamental property of scaling AI models. (paper)
  • 05/01: A brief history of LLaMA models (tweet, blog)
  • 05/01: Geoffrey Hinton left Google. IBM say it can replace over 7500 current employees with AI. Chegg stock price drop 40%.
  • 04/30: PandaLM, provide reproducible and automated comparisons between different large language models (LLMs). (tweet, repo)
  • 04/30: Otter, a Multi-modal chatbots learn to perform tasks through rich instructions on media content (tweet, repo)
  • 04/30: Linly-ChatFlow, Shenzhen University release Linly-ChatFlow-7B/13B/33B/65B fintune on pre-trained Chinese-LLaMA with english and chinese intruction dataset (repo)
  • 04/29: MLC-LLM, an open framework that brings LLMs directly into a broad class of platforms (iPhone, CUDA, Vulkan, Metal) with GPU acceleration! (tweet, blog repo)
  • 04/29: Lamini: The LLM engine for rapidly customizing models without spinning up any GPUs (tweet, blog, repo, doc)
  • 04/29: FastChat-T5, a compact and commercial-friendly chatbot, Fine-tuned from Flan-T5, Outperforms Dolly-V2 with 4x fewer parameters (tweet, repo)
  • 04/29: StabilityAI/StableVicuna, Carper AI from StabilityAI family release RLHF-trained version of Vicuna-13B! (tweet, blog, model)
  • 04/29: StabilityAI/DeepFloyd IF, a powerful text-to-image model that can smartly integrate text into images, utilizes T5-XXL-1.1 as text encoder (tweet, blog)
  • 04/29: MosaicML/SD2, Training Stable Diffusion from Scratch for <$50k with MosaicML (tweet, blog)
  • 04/29: gpt4free, use gpt-4/3.5 free from sites (repo)
  • 04/29: OpenRL is an open-source general reinforcement learning research framework that supports training for various tasks such as single-agent, multi-agent, and natural language. Developed based on PyTorch by chinese company 4paradigm (repo)
  • 04/28: Chinese-LLaMA-Plus-7B, re-pretrain LLaMA on larger(120G) general corpus, fine-tune with 4M instruction dataset, bigger LoRA rank for less precision loss, beat former 13B mdoel on benchmark (repo)
  • 04/28: AudioGPT, a multi-modal GPT model can understand audio/text/image instruction inputs and generate audio, song, style transfer speech, talking head synthesis video (blog, repo, demo)
  • 04/28: Multimodal-GPT, released by the famous MMLab, build base on open-source multi-modal model OpenFlamingo with visual and language instructions (repo)
  • 04/27: "Speed is all you need", generate a 512 × 512 image with 20 iterations on GPU equipped mobile devices in 12- seconds for Stable Diffusion 1.4 without INT8 quantization, 50+% latency reduced on Samsung S23 Ultra. (paper)
  • 04/27: replit-code-v1-3b, it's a 2.7B parameters LLM trained entirely on code in 10 days, performs 40% better than comparable models on benchmark (tweet, model)
  • 04/26: LaMini-LM, a diverse set of 15 (more coming) mini-sized models (up to 1.5B) distilled from 2.6M instructions, comparable in performance to Alpaca-7B in downstream NLP + human eval (tweet, repo, data)
  • 04/26: huggingChat, a 30B OpenAssistant/oasst-sft-6-llama-30b-xor LLM deployed by huggingface (tweet, site, model)
  • 04/26: LLM+P, takes in a planning problem decription, turn it into PDDL, leveraging classical planners to find a solution (tweet, paper, repo)
  • 04/25: NeMo Guardrails, the new toolkit for easily developing trustworthy LLM-based conversational applications (tweet)
  • 04/21: China Fudan University release its 16B LLM named MOSS-003; Moss dataset contains ~1.1M text-davinci-003 generated self-instruct dataset, include ~300k plugins dataset as text-to-image/equations/.etc, fp16 finetune on 2 A100s or 4/8-bit finetune on single 3090. (repo)
  • 04/21: Phoenix, a new multilingual LLM that achieves competitive performance, vast collection of popular open source dataset (repo)
  • 04/20: UltraChat, a Informative and Diverse Multi-round Chat Data gather by THUNLP lab (repo, data)
  • 04/20: replicate ChatGLM with efficient fine-tuning (ptunig, lora, freeze) (repo); support langchain in langchain-ChatGLM project
  • 04/19: StableLM, 3B/7B LLM from StabilityAI (tweet, blog)
  • 04/18: Semantic Kernel, MSFT release its contextual memory tool like langchain/gptindex (repo)
  • 04/17: LLaVA: Large Language and Vision Assistant, Visual Instruction Tuning (blog, repo, demo)
  • 04/17: MiniGPT-4, multi-modal LLM like GPT4, consists of a vision encoder with a pretrained ViT and Q-Former, a single linear projection layer, and an advanced Vicuna large language model (blog, repo)
  • 04/17: TogetherCompute/RedPajama, reproduce LLaMA with 1.2 trillion tokens (blog, tweet)
  • 04/16: LAION-AI/Open-Assistant, is an open-source chat model(includes datasets: consists of a ~161K human-annotated assistant-style conversation corpus, including 35 different languages and annotated with ~461K quality ratings) (tweet, repo, mdoels)
  • 04/15: WebLLM, an open-source chatbot that brings LLMs like Vicuna directly onto web browsers (tweet, blog, repo)
  • 04/12: Dolly-v2-12b, Databricks release its open source Dolly-v2-12b model, derived from EleutherAI’s Pythia-12b and fine-tuned on a ~15K record instruction corpus generated by Databricks employees, which is open source as well (blog, repo, model)
  • 04/12: DeepSpeed Chat, DeepSpeed from MSFT support RLHF fine-tune with affordable haraware (blog)
  • 04/12: Text-to-SQL from self-debugging explanation component (tweet)
  • 04/11: AgentGPT, generative agents were able to simulate human-like behavior in an interactive sandbox (tweet)
  • 04/11: AutoGPT, autonomously achieve whatever goal you set (repo)
  • 04/11: Raven v8 14B released (tweet, model, repo)
  • 04/09: SVDiff, diffusion fine-tune method smaller than LoRA (tweet, repo)
  • 04/09: RPTQ, new 3 bit quantization (repo, paper)
  • 04/08: Wonder Studio, robot beat human with kongfu (tweet)
  • 04/08: chatGDB, chatgpt for GDB (tweet, repo)
  • 04/08: Vicuna-7B, small yet capable (repo), Vicuna shows impressive performance against GPT4 by lastest paper of MSFTResearch (tweet)
  • 04/07: Instruction tuning with GPT4, academic self-instruct guide from microsoft research (tweet, blog, repo, paper)
  • 04/07: ChatPipe, Orchestrating Data Preparation Program by Optimizing Human-ChatGPT Interactions (blog)
  • 04/07: Chinese-LLaMA-Alpaca release its 13B model (tweet)
  • 04/07: MathPrompter, How to chatwith GPT3 Davinci API and archive better on math benmark (paper)
  • 04/07: engshell, interact with your shell using english language (tweet)
  • 04/07: a chinese geek fine-tune a chatglm-6b model on his wechat dialogue and blog to produce a digital version of him self (tweet)
  • 04/06: StackLLaMA, A hands-on guide to train LLaMA with RLHF, fine-tuned on stack exchange QA data (tweet, blog, demo)
  • 04/06: Arxiv Chat, chat with the lastest papers (tweet)
  • 04/06: Firefly, a 1.4B/2.6B chinese chat LLM, finetune on 1.1M multi-task dataset (repo)
  • 04/06: a chinese guide of chatgpt repo
  • 04/06: LamaCleaner, segment anything and inpaint anything (tweet)
  • 04/05: SAM, Meta AI release Segment Anything Model as foundation model for image segmentation, and SA-1B dataset, which is 400x larger than any existing segmentation dataset (tweet)
  • 04/04: a beautiful cli for chatgpt (tweet)
  • 04/04: Baize, fine-tune with LoRA using 100K dialogs ChatGPT self-chat and other opensource dataset, released 7B, 13B and 30B models (repo, tweet, demo, model)
  • 04/03: Koala-13B, fine-tuned from LLaMA on user-shared conversations and open-source datasets, performs similarly to Vicuna (blog, demo, repo)
  • 04/02: LMFlow, train on single 3090 for 5 hours and get your own chatgpt (blog, repo)
  • 04/01: Alpaca-CoT, extend CoT data to Alpaca to boost its reasoning ability, provide gathered datasets (repo)
  • 04/01: Vicuna-13B, An Open-Source Chatbot Impressing GPT-4 with 90% ChatGPT Quality, fine-tune LLaMA on ~70K conversations from ShareGPT (blog, repo, demo, data, gptq-4-bit)
  • 04/01: Twitter's Recommendation Algorithm (repo)
  • 04/01: PolyglotSiri Apple Shortcut, (tweet, repo)
  • 04/01: Official Apple Core ML Stable Diffusion Library, M-series chips beat 4090, (repo, MochiDiffusion, swift-coreml-diffusers)
  • 03/31: BloombergGPT, 50B LLM outperform existing models on financial tasks (tweet)
  • 03/31: HuggingGPT, as an interface for LLMs to connect AI Models for solving comlicated AI tasks (tweet, demo)
  • 03/31: Llama-X (repo)
  • 03/31: GPT4 UI generation (tweet)
  • 03/30: ChatExplore (tweet)
  • 03/30: ColossalChat, from ColossalAI (demo, tweet, medium, repo, serve)
  • 03/30: ChatGLM-6B, from THUDM(Tsinghua University), code and data not release (repo, model)
  • 03/29: Uncle Rabbit, the first conversational holographic AI (tweet, blog)
  • 03/29: chatgpt instead of siri (tweet)
  • 03/29: LLaMA-Adapter, fine-tuning LLaMA with 1.2M learnable parameters in 1 hour on 8 A100 (tweet, repo, demo)
  • 03/28: Chinese-LLaMA-Alpaca, add 20K chinese sentencepiece tokens to vocab and pre-trained LLaMA in 2 steps, fine-tuned LLaMA on a 2M chinese corpus using Alpaca-LoRA, 7B model released, dataset not (repo, tweet, blog, model)
  • 03/28: gpt4all, fine-tune LLaMa using LoRA with ~800k gpt3.5-turbo generations, include clean assistant data including code, stories and dialogue (repo, model, data)
  • 03/24: Dolly, Databricks fine-tune alpaca dataset on gpt-j-6b (repo)
  • 03/22: Alpaca-LoRA-Serve, gradio based chatbot service (tweet, repo)
  • 03/22: Alpaca-LoRA, reproducing the Stanford Alpaca results using low-rank adaptation(LoRA) on RTX4090 and run on a Raspberry Pi 4 (tweet, repo, demo, model, blog, reproduce tweet, zhihu, sina, explain)
  • 03/22: BELLE, fine-tune BLOOMZ-7B1-mt and LLaMA(7B/13B) on a 1.5M chinese dataset generate in a alpaca way, (repo, model)
  • 03/17: instruct-gpt-j, NLPCloud team fine-tune GPT-J using Alpaca's dataset (blog, model)
  • 03/13: Stanford Alpaca, fine-tune LLaMA 7B with a 52K single-turn instruction-followling dataset generate from OpenAI’s text-davinci-003 (blog, repo)
  • 03/11: ChatIE, solving Zero-Shot Information Extraction problem by enhancing ChatGPT with CoT prompting, achieve good performance on primary IE benchmarks (repo)
  • prompt engineering guide (blog), openai best practices (blog), prompt prefect (blog), prompt searching (repo), PromptInject (repo), auto prompt engineering (blog)
  • peft: State-of-the-art Parameter-Efficient Fine-Tuning (PEFT) methods (repo)
  • GPTQ-for-LLaMa: 4 bits quantization of LLaMA using GPTQ (repo)
  • llama.cpp: Inference of LLaMA model in pure C/C++, support different hardware platform & models, support 4-bit quantization using ggml format (repo, alpaca.cpp); support python bindings (llama-cpp-python, pyllamacpp, llamacpp-python )
  • llama_index: connect LLM with external data (repo), like langchain (repo)
  • llama-dl: high speed download of LLaMA model (repo(deprecated), model)
  • text-generation-webui: A gradio web UI for deploy LLMs like GPT-J, LLaMA (repo)
  • tldream/lama-cleaner: tiny little diffusion drawing app (repo1, repo2)
  • A1111-Web-UI-Installer: A gradio web UI for deploy stable diffusion models (repo)