/high-dim-universality-erm

Numerical experiments exploring the "universality" of empirical risk minimization in high dimensions.

Primary LanguageHTMLMIT LicenseMIT

Universality of Empirical Risk Minimization

Numerical experiments exploring the "universality" of empirical risk minimization (ERM) in high dimensions.

Recent work in high-dimensional statisitcs reveals a phenomonon where complex non-linear models have equivalent training and generalization error to corresponding linear gaussian models in the high-dimensional asymptotics $n,p \to \infty, \ \frac{p}{n} \to \gamma \in (0, \infty)$. Here, we explore whether this universality phenomenon holds in settings more general than the ones already studied.

Relevant papers:

Universality Laws for High-Dimensional Learning with Random Features --- Hong Hu & Yue M. Lu (2022)

Universality of Empirical Risk Minimization --- Andrea Montanari & Basil Saeed (2022)