EngineerKhan/Stanford-AI-Alignment-Double-Descent-Tutorial

Code for Arxiv Preprint "Double Descent Demystified: Identifying, Interpreting & Ablating the Sources of a Deep Learning Puzzle"

Double Descent Demystified

Identifying, Interpreting & Ablating the Sources of a Deep Learning Puzzle

This repository contains the code and data for our preprint "Double Descent Demystified: Identifying, Interpreting & Ablating the Sources of a Deep Learning Puzzle".

For a step-by-step explanation, see the walkthrough. The walkthrough contains mathematical intuition via ordinary linear regression, visual intuition via polynomial regression, and ablations in linear regression on real data.

• For ordinary linear regression on real and synthetic data, see linear_regression.py.
• For ordinary linear regression on real and synthetic data with ablations of double descent, see linear_regression_ablations.py
• For polynomial regression on synthetic data, see polynomial_regression.py.

Authorship

Authors: Rylan Schaeffer, Mikail Khona, Zachary Robertson, Akhilan Boopathy, Kateryna Pistunova, Jason W. Rocks, Ila Rani Fiete, Sanmi Koyejo.