PyTorch implementation of our matrix-multiplication-only KFAC based on Simplifying Momentum-based Positive-definite Submanifold Optimization with Applications to Deep Learning (ICML 2023)
-
Download the dataset from https://www.kaggle.com/datasets/ambityga/imagenet100
-
The dataset should be saved in the data folder (i.e., data/imagenet100/)
-
You can reproduce the results on imagenet-100 by using the provided bash scripts
-
Todo:
- Add code for optimization on SPD manifolds