Double Descent Demystified
Identifying, Interpreting & Ablating the Sources of a Deep Learning Puzzle
This repository contains the code and data for our preprint "Double Descent Demystified: Identifying, Interpreting & Ablating the Sources of a Deep Learning Puzzle".
For a step-by-step explanation, see the walkthrough. The walkthrough contains mathematical intuition via ordinary linear regression, visual intuition via polynomial regression, and ablations in linear regression on real data.
- For ordinary linear regression on real and synthetic data, see linear_regression.py.
- For ordinary linear regression on real and synthetic data with ablations of double descent, see linear_regression_ablations.py
- For polynomial regression on synthetic data, see polynomial_regression.py.
Authorship
Authors: Rylan Schaeffer, Mikail Khona, Zachary Robertson, Akhilan Boopathy, Kateryna Pistunova, Jason W. Rocks, Ila Rani Fiete, Sanmi Koyejo.