Some Materials about SP 18 course
-
The first homework will be a comprehensive examples on dimension reduction. kernel PCA/CCA Renyi Correlation(ACE) IsoMap, tSNE. Some practical examples on that.
-
Latent is continuous and discrete, Mixture of Gaussian, Tensor Method package: tensorly (1) HMM on tensor method. (2) Mixture of Gaussian on tensor method. (3) EM convergence proof.
-
Modern Unsupervised Learning: Ganerative Models The 4th module will be on AE and GAN. AE: AE, VAE, AAE, WAE. GAN: WGAN, ConditionalGAN, DualGAN.
The second homework will be on using Kaggle's nuclear weapon: XGBoost and Random Forest.
-
Boosting(proof, and how it works,,), CART. Random Forest. XGBoost, as homework. also proofs.
-
Mixture of expert.
The third homework wil be on basic neural networks, FC-NN, simple CNN, simple RNN.
Also something about optimizations as homework example.
Architecture: NN: CNN+ pooling, Inception v2,v3, v4.
BP & Architecture, loss function design, training algorithms. (1)Training(optimization), Rahul, SGD++, gradient flow. (2)Proof of representation learning: (tensor, landscape design(GLU). ) (3) Escaping saddle point. (sanjeev, jin chi, ) (4) generalization,
representation theorem of NN.
Homework:
practical: generative model: generate with NN, decode with NN. -> everywhere. figure out what it is.
Find the signature of the data, to decide the model.
Adversial Examples Interpretability Generalization Theory on Neural Networks Tensor/ Landscape Design Graph Neural Networks
The final projects will be open. Could be (1) bio-related problem with Deep Learning (2) theoritical problems with Deep Learning.