Developed in collaboration with vug. The repo can calculate the Hessian matrix and/or its spectrum for simple neural nets. It can also find the largest and smallest few eigenvalues without finding the full Hessian.
Some results using this code has been used in the following work:
[1] Levent Sagun, Utku Evci, V. Ugur Guney, Yann Dauphin, Leon Bottou, Empirical Analysis of the Hessian of Over-Parametrized Neural Networks
Relevant related works are:
[2] Levent Sagun, Leon Bottou, Yann LeCun, Eigenvalues of the Hessian in Deep Learning: Singularity and Beyond
[3] Pratik Chaudhari, Anna Choromanska, Stefano Soatto, Yann LeCun, Carlo Baldassi, Christian Borgs, Jennifer Chayes, Levent Sagun, Riccardo Zecchina, Entropy-SGD: Biasing Gradient Descent Into Wide Valleys