language-model
There are 1944 repositories under language-model topic.
huggingface/transformers
🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.
xtekky/gpt4free
The official gpt4free repository | various collection of powerful language models
microsoft/generative-ai-for-beginners
18 Lessons, Get Started Building with Generative AI 🔗 https://microsoft.github.io/generative-ai-for-beginners/
dair-ai/Prompt-Engineering-Guide
🐙 Guides, papers, lecture, notebooks and resources for prompt engineering
LAION-AI/Open-Assistant
OpenAssistant is a chat-based assistant that understands tasks, can interact with third-party systems, and retrieve information dynamically to do so.
tatsu-lab/stanford_alpaca
Code and documentation to train Stanford's Alpaca models, and generate the data.
hiyouga/LLaMA-Factory
Unify Efficient Fine-Tuning of 100+ LLMs
mlc-ai/mlc-llm
Universal LLM Deployment Engine with ML Compilation
deepset-ai/haystack
:mag: LLM orchestration framework to build customizable, production-ready LLM applications. Connect components (models, vector DBs, file converters) to pipelines or agents that can interact with your data. With advanced retrieval methods, it's best suited for building RAG, question answering, semantic search or conversational agent chatbots.
BlinkDL/RWKV-LM
RWKV is an RNN with transformer-level LLM performance. It can be directly trained like a GPT (parallelizable). So it's combining the best of RNN and transformer - great performance, fast inference, saves VRAM, fast training, "infinite" ctx_len, and free sentence embedding.
mlc-ai/web-llm
High-performance In-browser LLM Inference Engine
microsoft/LoRA
Code for loralib, an implementation of "LoRA: Low-Rank Adaptation of Large Language Models"
BlinkDL/ChatRWKV
ChatRWKV is like ChatGPT but powered by RWKV (100% RNN) language model, and open source.
brightmart/nlp_chinese_corpus
大规模中文自然语言处理语料 Large Scale Chinese Corpus for NLP
mlfoundations/open_clip
An open source implementation of CLIP.
huggingface/tokenizers
💥 Fast State-of-the-Art Tokenizers optimized for Research and Production
vercel/ai
Build AI-powered applications with React, Svelte, Vue, and Solid
EleutherAI/gpt-neo
An implementation of model parallel GPT-2 and GPT-3-style models using the mesh-tensorflow library.
speechbrain/speechbrain
A PyTorch-based Speech Toolkit
OptimalScale/LMFlow
An Extensible Toolkit for Finetuning and Inference of Large Foundation Models. Large Models for All.
neuml/txtai
💡 All-in-one open-source embeddings database for semantic search, LLM orchestration and language model workflows
EleutherAI/gpt-neox
An implementation of model parallel autoregressive transformers on GPUs, based on the Megatron and DeepSpeed libraries
OpenNMT/OpenNMT-py
Open Source Neural Machine Translation and (Large) Language Models in PyTorch
codertimo/BERT-pytorch
Google AI 2018 BERT pytorch implementation
ramon-victor/freegpt-webui
GPT 3.5/4 with a Chat Web UI. No API key required.
EleutherAI/lm-evaluation-harness
A framework for few-shot evaluation of language models.
THUDM/CogVLM
a state-of-the-art-level open visual language model | 多模态预训练模型
aiwaves-cn/agents
An Open-source Framework for Autonomous Language Agents
CLUEbenchmark/CLUE
中文语言理解测评基准 Chinese Language Understanding Evaluation Benchmark: datasets, baselines, pre-trained models, corpus and leaderboard
yizhongw/self-instruct
Aligning pretrained language models with instruction data generated by themselves.
ravenscroftj/turbopilot
Turbopilot is an open source large-language-model based code completion engine that runs locally on CPU
JohnSnowLabs/spark-nlp
State of the Art Natural Language Processing
xlang-ai/OpenAgents
OpenAgents: An Open Platform for Language Agents in the Wild
hiyouga/ChatGLM-Efficient-Tuning
Fine-tuning ChatGLM-6B with PEFT | 基于 PEFT 的高效 ChatGLM 微调
mlfoundations/open_flamingo
An open-source framework for training large multimodal models.
eth-sri/lmql
A language for constraint-guided and efficient LLM programming.