pretrained-language-model

There are 131 repositories under pretrained-language-model topic.

  • wenge-research/YAYI2

    YAYI 2 是中科闻歌研发的新一代开源大语言模型，采用了超过 2 万亿 Tokens 的高质量、多语言语料进行预训练。(Repo for YaYi 2 Chinese LLMs)

    Language:Python3.4k71019

  • microsoft/torchscale

    Foundation Architecture for (M)LLMs

    Language:Python3.1k4486219

  • Separius/awesome-sentence-embedding

    A curated list of pretrained sentence and word embedding models

    Language:Python2.3k7719262

  • THUDM/P-tuning-v2

    An optimized deep prompt tuning strategy comparable to fine-tuning across scales and tasks

    Language:Python2.1k2978204

  • thunlp/OpenDelta

    A plug-and-play library for parameter-efficient-tuning (Delta Tuning)

    Language:Python1k196483

  • xcfcode/Summarization-Papers

    Summarization Papers

    Language:TeX1k233146

  • AndrewZhe/lawyer-llama

    中文法律LLaMA (LLaMA for Chinese legel domain)

    Language:Python9571167131

  • gaoisbest/NLP-Projects

    word2vec, sentence2vec, machine reading comprehension, dialog system, text classification, pretrained language model (i.e., XLNet, BERT, ELMo, GPT), sequence labeling, information retrieval, information extraction (i.e., entity, relation and event extraction), knowledge graph, text generation, network embedding

    Language:OpenEdge ABL558211153

  • allenai/dont-stop-pretraining

    Code associated with the Don't Stop Pretraining ACL 2020 paper

    Language:Python53583973

  • OpenBMB/CPM-Live

    Live Training for Open-source Big Models

    Language:Python507203839

  • RenzeLou/awesome-instruction-learning

    Papers and Datasets on Instruction Tuning and Following. ✨✨✨

    Language:Python4997025

  • Hzfinfdu/Diffusion-BERT

    ACL'2023: DiffusionBERT: Improving Generative Masked Language Models with Diffusion Models

    Language:Python324113326

  • LYH-YF/MWPToolkit

    MWPToolkit is an open-source framework for math word problem(MWP) solvers.

    Language:Python16332637

  • yueyu1030/AttrPrompt

    [NeurIPS 2023] This is the code for the paper `Large Language Model as Attributed Training Data Generator: A Tale of Diversity and Bias`.

    Language:Python1533613

  • zzz47zzz/awesome-lifelong-learning-methods-for-llm

    [ACM Computing Surveys 2025] This repository collects awesome survey, resource, and paper for Lifelong Learning with Large Language Models. (Updated Regularly)

    146316

  • hyintell/awesome-refreshing-llms

    EMNLP'23 survey: a curation of awesome papers and resources on refreshing large language models (LLMs) without expensive retraining.

    1345011

  • ZhengZixiang/ATPapers

    Worth-reading papers and related resources on attention mechanism, Transformer and pretrained language model (PLM) such as BERT. 值得一读的注意力机制、Transformer和预训练语言模型论文与相关资源集合

    1337013

  • DC-research/TEMPO

    The official code for "TEMPO: Prompt-based Generative Pre-trained Transformer for Time Series Forecasting (ICLR 2024)". TEMPO is one of the very first open source Time Series Foundation Models for forecasting task v1.0 version.

    Language:Python12021414

  • microsoft/COCO-LM

    [NeurIPS 2021] COCO-LM: Correcting and Contrasting Text Sequences for Language Model Pretraining

    Language:Python1173713

  • SuperBruceJia/Awesome-LLM-Self-Consistency

    Awesome LLM Self-Consistency: a curated list of Self-consistency in Large Language Models

    108417

  • git-disl/BERT4ETH

    BERT4ETH: A Pre-trained Transformer for Ethereum Fraud Detection (WWW23)

    Language:Python10251120

  • thunlp/Prompt-Transferability

    On Transferability of Prompt Tuning for Natural Language Processing

    Language:Python1006811

  • SJTU-IPADS/Bamboo

    Bamboo-7B Large Language Model

    931011

  • RUCAIBox/UniCRS

    [KDD22] Official PyTorch implementation for "Towards Unified Conversational Recommender Systems via Knowledge-Enhanced Prompt Learning".

    Language:Python871015

  • yumeng5/TopClus

    [WWW 2022] Topic Discovery via Latent Space Clustering of Pretrained Language Model Representations

    Language:Python872611

  • GanjinZero/CODER

    CODER: Knowledge infused cross-lingual medical term embedding for term normalization. [JBI, ACL-BioNLP 2022]

    Language:Python79155

  • EagleW/Scientific-Inspiration-Machines-Optimized-for-Novelty

    Official implementation of the ACL 2024: Scientific Inspiration Machines Optimized for Novelty

    Language:Python772211

  • ChangwenXu98/TransPolymer

    Implementation of "TransPolymer: a Transformer-based language model for polymer property predictions" in PyTorch

    Language:Python6821421
  • Awesome-LLMs-ICLR-24

    azminewasi/Awesome-LLMs-ICLR-24

    It is a comprehensive resource hub compiling all LLM papers accepted at the International Conference on Learning Representations (ICLR) in 2024.

    64103

  • yumeng5/SuperGen

    [NeurIPS 2022] Generating Training Data with Language Models: Towards Zero-Shot Language Understanding

    Language:Python642314

  • FranxYao/PoincareProbe

    Implementation of ICLR 21 paper: Probing BERT in Hyperbolic Spaces

    Language:Jupyter Notebook58605

  • SKplanet/Dialog-KoELECTRA

    ELECTRA기반 한국어 대화체 언어모델

    Language:Python54737

  • GanjinZero/BioBART

    BioBART: Pretraining and Evaluation of A Biomedical Generative Language Model [ACL-BioNLP 2022]

    Language:Python52254

  • heraclex12/NLP2SPARQL

    Translate Natural Language Processing to SPARQL Query and vice versa

    Language:Python522612

  • zjukg/DUET

    [Paper][AAAI 2023] DUET: Cross-modal Semantic Grounding for Contrastive Zero-shot Learning

    Language:Python52338

  • OpenMatch/COCO-DR

    [EMNLP 2022] This is the code repo for our EMNLP‘22 paper "COCO-DR: Combating Distribution Shifts in Zero-Shot Dense Retrieval with Contrastive and Distributionally Robust Learning".

    Language:Python50444