In recent years, deep learning has made great progress. It is widely used in many fields, such as computer vision, speech recognition, natural language processing, bioinformatics, etc., and also has a significant impact on basic science fields, such as applied and computational mathematics, statistical physics, computational chemistry and materials science, life science, etc. However, it should be noted that deep learning as a black-box model, its ability is explored through a large number of experiments. The theory of deep learning has gradually attracted the attention of many researchers, and has made progress in many aspects.
This course is closely around the latest development of deep learning theory. It intends to teach mathematical models, theories, algorithms and numerical experiments related to a series of basic problems from multiple perspectives of deep learning theory. This course is designed for doctoral, postgraduate and senior undergraduate students in all majors, who have basic knowledge of machine learning and deep learning.
The topics and the corresponding material are as follows:
- Introduction to Deep Learning material
- Approximation Theory of Neural Networks material
- Algorithmic Regularization material
- Inductive Biases due to Algorithmic Regularization material
- Tractable Landscapes for Nonconvex Optimization material
- Neural Tangent Kernel material
- Multilayer Convolutional Sparse Coding material
- Information Bottleneck and Others material
- The Fragility of Neural Networks material
- Neural ODEs material
- Neural Networks Learn the Geodesic Curve in the Wasserstein Space material
- Generative Model：GAN, VAE and Roundtrip material
- Interpretability of Neural Networks, Frequency principle material
Mathematical Analysis, Linear Algebra, Mathematical Statistics, Numerical Optimization, Matrix Theory, Fundamentals of Machine Learning and Deep Learning