- 2006 (Yann LeCun) (CVPR) Dimensionality reduction by learning an invariant mapping
- 2015 (CMU) (ICCV) Unsupervised Learning of Visual Representations Using Videos
- 2015 (CMU) (ICCV) Unsupervised Visual Representation Learning by Context Prediction
- 2016 (CVPR) Slow and steady feature analysis - higher order temporal coherence in video
- 2016 (ECCV) Unsupervised learning of visual representations by solving jigsaw puzzles
- 2016 (UCB) (CVPR) Context Encoders - Feature Learning by Inpainting
- 2016 (UCB) (ECCV) Colorful image colorization
- 2017 (Google) (ICRA) Time-Contrastive Networks - Self-Supervised Learning from Video
- 2017 (ICCV) Unsupervised Representation Learning by Sorting Sequences
- 2017 (Kaiming He) (ICCV) Transitive Invariance for Self-Supervised Visual Representation Learning
- 2017 (UCB) (CVPR) Split-Brain Autoencoders - Unsupervised Learning by Cross-Channel Prediction
- 2018 (DeepMind) (Arxiv) [CPC] Representation Learning with Contrastive Predictive Coding
- 2018 (ICLR) Unsupervised Representation Learning by Predicting Image Rotations
- 2019 (Arxiv) [CPC2] Data-Efficient Image Recognition with Contrastive Predictive Coding
- 2019 (Facebook) (Arxiv) Scaling and Benchmarking Self-Supervised Visual Representation Learning
- 2019 (Google) (ICCV) VideoBERT - A Joint Model for Video and Language Representation Learning
- 2019 (NIPS) [AMDIM] Learning representations by maximizing mutual information across views
- 2019 (Yoshua Bengio) (ICLR) [DIM] Learning deep representations by mutual information estimation and maximization
- 2020 (DeepMind) (Arxiv) [BYOL] Bootstrap Your Own Latent - A New Approach to Self-Supervised Learning
- 2020 (Google) (Arxiv) Supervised Contrastive Learning
- 2020 (Hinton) (Arxiv) Big Self-Supervised Models are Strong Semi-Supervised Learners
- 2020 (Hinton) (ICML) [SimCLR] A Simple Framework for Contrastive Learning of Visual Representations
- 2020 (Kaiming He) (CVPR) [MoCo] Momentum Contrast for Unsupervised Visual Representation Learning
- 2013 (Google) (NIPS) [word2vec] Distributed representations of words and phrases and their compositionality
- 2017 (Google) (NIPS) [Transformer] Attention is All you Need
- 2018 (Google) (NAACL) [BERT] Bert - Pre-training of deep bidirectional transformers for language understanding
- 2018 (OpenAI) (Arxiv) [GPT] Improving Language Understanding by Generative Pre-Training
- 2019 (ACL) Self-Supervised Dialogue Learning
- 2019 (OpenAI) (Arxiv) [GPT2] Language Models are Unsupervised Multitask Learners
- 2020 (DeepMind) (ICLR) A Mutual Information Maximization Perspective of Language Representation Learning
- 2020 (OpenAI) (Arxiv) [GPT3] Language Models are Few-Shot Learners
- 2020 (UW) (ACL) Don’t Stop Pretraining - Adapt Language Models to Domains and Tasks
- 2020 (Arxiv) Self-supervised Learning on Graphs - Deep Insights and New Direction
- 2020 (KDD) GPT-GNN - Generative Pre-Training of Graph Neural Networks
- 2020 (WWW) Graph Representation Learning via Graphical Mutual Information Maximization
- 2020 (CIKM) [S3Rec] S3-Rec - Self-Supervised Learning for Sequential Recommendation with Mutual Information Maximization
- 2020 (Google) (Arxiv) Self-supervised Learning for Large-scale Item Recommendations
- 2020 (SIGIR) Self-Supervised Reinforcement Learning for Recommender Systems
- 2020 (Xiangnan He) (Arxiv) Self-supervised Graph Learning for Recommendation
- 2017 (DeepMind) (ICCV) Multi-task Self-Supervised Visual Learning
- 2018 (CVPR) Cross-domain Self-supervised Multi-task Feature Learning Using Synthetic Imagery
- 2019 (Facebook) (Arxiv) Billion-scale semi-supervised learning for image classification
- 2019 (Google) (ICCV) S4L - Self-Supervised Semi-Supervised Learning
- 2020 (Google) (CVPR) Self-Training With Noisy Student Improves ImageNet Classification
- 2020 (Google) (NIPS) Rethinking Pre-training and Self-training
- 2019 (Arxiv) Rethinking Data Augmentation - Self-Supervision and Self-Distillation
- 2019 (ICCV) Boosting Few-Shot Visual Learning with Self-Supervision