This repo contains the code of the following 6 papers:
Understanding the Role of Nonlinearity in Training Dynamics of Contrastive Learning
Yuandong Tian
Understanding Deep Contrastive Learning via Coordinate-wise Optimization
Yuandong Tian
NeurIPS'22 Oral
Understanding self-supervised Learning Dynamics without Contrastive Pairs
Yuandong Tian, Xinlei Chen, Surya Ganguli
ICML 2021 link, Outstanding Paper Honorable Mention
Understanding Self-supervised Learning with Dual Deep Networks
Yuandong Tian, Lantao Yu, Xinlei Chen, Surya Ganguli
arXiv link
Student Specialization in Deep ReLU Networks With Finite Width and Input Dimension (./student_specialization)
Yuandong Tian
ICML 2020 link
Luck Matters: Luck Matters: Understanding Training Dynamics of Deep ReLU Networks (./luckmatter)
Yuandong Tian, Tina Jiang, Qucheng Gong, Ari Morcos
arxiv link