Tarpelite/BUAA-PTM

BUAA-awesome-PTM

A curated list of resources for self-supervised pre-trained models (PTM), especially in attack and defense.

contents列表

模型架构 (Architecture)

1. NLP (Natural Language Processing)

• Bert: Pre-training of deep bidirectional transformers for language understanding

• RoBERTa: A Robustly Optimized BERT Pretraining Approach

2. CV （Computer Vision)

• Moco:Momentum Contrast for Unsupervised Visual Representation Learning

• SimCLR:A Simple Framework for Contrastive Learning of Visual Representations

3. Speech (Speech)

• SpeechBERT: An Audio-and-text Jointly Learned Language Model for End-to-end Spoken Question Answering

开源代码、工具、平台(Open Source Code & Tools & Platform)

• 1. 🤗 HuggingFace's Transformers: State-of-the-art Natural Language Processing by Huggingface Co. BERT家族的pytorch封装

• 2. fairseq: A Fast, Extensible Toolkit for Sequence Modeling by FaceBook， 预训练的序列模型的pytorch实现

攻击方法 ( Attack Method)

• 1. TextFooler：Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailment

• 2. Extracting Training Data from Large Language Models

• 3. Membership Inference Attacks against Machine Learning Models

• 4. BERT-ATTACK: Adversarial Attack Against BERT Using BERT

防御方法 (Defend Method)

• 1. TextHide: Tackling Data Privacy in Language Understanding Tasks

漏洞报道 (Defeacts Report)

• 1. Privacy Considerations in Large Language Models

• 2. GPT3之语言垃圾