pretrained-language-model
There are 143 repositories under pretrained-language-model topic.
wenge-research/YAYI2
YAYI 2 是中科闻歌研发的新一代开源大语言模型，采用了超过 2 万亿 Tokens 的高质量、多语言语料进行预训练。(Repo for YaYi 2 Chinese LLMs)
microsoft/torchscale
Foundation Architecture for (M)LLMs
Separius/awesome-sentence-embedding
A curated list of pretrained sentence and word embedding models
THUDM/P-tuning-v2
An optimized deep prompt tuning strategy comparable to fine-tuning across scales and tasks
thunlp/OpenDelta
A plug-and-play library for parameter-efficient-tuning (Delta Tuning)
xcfcode/Summarization-Papers
Summarization Papers
AndrewZhe/lawyer-llama
中文法律LLaMA (LLaMA for Chinese legel domain)
gaoisbest/NLP-Projects
word2vec, sentence2vec, machine reading comprehension, dialog system, text classification, pretrained language model (i.e., XLNet, BERT, ELMo, GPT), sequence labeling, information retrieval, information extraction (i.e., entity, relation and event extraction), knowledge graph, text generation, network embedding
allenai/dont-stop-pretraining
Code associated with the Don't Stop Pretraining ACL 2020 paper
OpenBMB/CPM-Live
Live Training for Open-source Big Models
RenzeLou/awesome-instruction-learning
Papers and Datasets on Instruction Tuning and Following. ✨✨✨
rednote-hilab/dots.llm1
The official repository of the dots.llm1 base and instruct models proposed by rednote-hilab.
Hzfinfdu/Diffusion-BERT
ACL'2023: DiffusionBERT: Improving Generative Masked Language Models with Diffusion Models
LYH-YF/MWPToolkit
MWPToolkit is an open-source framework for math word problem(MWP) solvers.
yueyu1030/AttrPrompt
[NeurIPS 2023] This is the code for the paper `Large Language Model as Attributed Training Data Generator: A Tale of Diversity and Bias`.
zzz47zzz/awesome-lifelong-learning-methods-for-llm
[ACM Computing Surveys 2025] This repository collects awesome survey, resource, and paper for Lifelong Learning with Large Language Models. (Updated Regularly)
hyintell/awesome-refreshing-llms
EMNLP'23 survey: a curation of awesome papers and resources on refreshing large language models (LLMs) without expensive retraining.
ZhengZixiang/ATPapers
Worth-reading papers and related resources on attention mechanism, Transformer and pretrained language model (PLM) such as BERT. 值得一读的注意力机制、Transformer和预训练语言模型论文与相关资源集合
DC-research/TEMPO
The official code for "TEMPO: Prompt-based Generative Pre-trained Transformer for Time Series Forecasting (ICLR 2024)". TEMPO is one of the very first open source Time Series Foundation Models for forecasting task v1.0 version.
microsoft/COCO-LM
[NeurIPS 2021] COCO-LM: Correcting and Contrasting Text Sequences for Language Model Pretraining
git-disl/BERT4ETH
BERT4ETH: A Pre-trained Transformer for Ethereum Fraud Detection (WWW23)
SuperBruceJia/Awesome-LLM-Self-Consistency
Awesome LLM Self-Consistency: a curated list of Self-consistency in Large Language Models
thunlp/Prompt-Transferability
On Transferability of Prompt Tuning for Natural Language Processing
SJTU-IPADS/Bamboo
Bamboo-7B Large Language Model
yumeng5/TopClus
[WWW 2022] Topic Discovery via Latent Space Clustering of Pretrained Language Model Representations
RUCAIBox/UniCRS
[KDD22] Official PyTorch implementation for "Towards Unified Conversational Recommender Systems via Knowledge-Enhanced Prompt Learning".
EagleW/Scientific-Inspiration-Machines-Optimized-for-Novelty
Official implementation of the ACL 2024: Scientific Inspiration Machines Optimized for Novelty
ChangwenXu98/TransPolymer
Implementation of "TransPolymer: a Transformer-based language model for polymer property predictions" in PyTorch
GanjinZero/CODER
CODER: Knowledge infused cross-lingual medical term embedding for term normalization. [JBI, ACL-BioNLP 2022]
yumeng5/SuperGen
[NeurIPS 2022] Generating Training Data with Language Models: Towards Zero-Shot Language Understanding
qianlima-lab/awesome-lifelong-learning-methods-for-llm
This repository collects awesome survey, resource, and paper for Lifelong Learning for Large Language Models. (Updated Regularly)
azminewasi/Awesome-LLMs-ICLR-24
It is a comprehensive resource hub compiling all LLM papers accepted at the International Conference on Learning Representations (ICLR) in 2024.
FranxYao/PoincareProbe
Implementation of ICLR 21 paper: Probing BERT in Hyperbolic Spaces
SKplanet/Dialog-KoELECTRA
ELECTRA기반 한국어 대화체 언어모델
heraclex12/NLP2SPARQL
Translate Natural Language Processing to SPARQL Query and vice versa
zzz47zzz/codebase-for-incremental-learning-with-llm
[ACL2024] A Codebase for Incremental Learning with Large Language Models; Official released code for "Learn or Recall? Revisiting Incremental Learning with Pre-trained Language Models (ACL 2024)", "Incremental Sequence Labeling: A Tale of Two Shifts (ACL 2024 Findings)", and "Concept-1K: A Novel Benchmark for Instance Incremental Learning (arxiv)"