Code for paper Emergence of hierarchical modes from deep learning(arXiv: 2208.09859). Here, given the current revolution of AI techniques (e.g., Chat GPT), a new learning framework named mode decomposition learning (MDL) is introduced, calling for a rethinking of conventional weight-based deep learning through the lens of cheap and interpretable mode-based learning. MDL explains the network performance with the leading modes, displaying a striking piecewise power-law behavior.
Python 3.8
- Each folder contains the relevant .py files for the figures in the main text, and the dataset folder is empty(due to the large size of data files), so you have to load the dataset yourself.
- Fig3 folder contains the .ipynb file for the figure, and the models are the pre-saved ones using model_save package in the Utils folder.
THE MNIST DATABASE of handwritten digits.
This code is the product of work carried out by the group of PMI lab, Sun Yat-sen University. If the code helps, consider giving us a shout-out in your publications.
If you have any question, please contact me via lich89@mail2.sysu.edu.cn.