Kirito-E2D/DeepLearning

CS583: Deep Learning

1. Machine learning basics. This part briefly introduces the fundamental ML problems-- regression, classification, dimensionality reduction, and clustering-- and the traditional ML models and numerical algorithms for solving the problems.

   • ML basics. [slides-1] [slides-2]

   • Regression. [slides-1] [slides-2]

   • Classification.

     • Logistic regression: [slides] [lecture note]

     • SVM: [slides]

     • Softmax classifier: [slides]

     • KNN classifier: [slides]

   • Regularizations. [slides-1] [slides-2]

   • Clustering. [slides]

   • Dimensionality reduction. [slides-1] [slides-2] [lecture note]

   • Scientific computing libraries. [slides]

2. Neural network basics. This part covers the multilayer perceptron, backpropagation, and deep learning libraries, with focus on Keras.

   • Multilayer perceptron and backpropagation. [slides] [lecture note]

   • Keras. [slides]

   • Further reading:

     • [activation functions]

     • [parameter initialization]

     • [optimization algorithms]

3. Convolutional neural networks (CNNs). This part is focused on CNNs and its application to computer vision problems.

   • CNN basics. [slides]

   • Tricks for improving test accuracy. [slides]

   • Feature scaling and batch normalization. [slides]

   • Advanced topics on CNNs. [slides]

   • Popular CNN architectures. [slides]

   • Face recognition. [slides]

   • Further reading:

     • [style transfer (Section 8.1, Chollet's book)]

     • [visualize CNN (Section 5.4, Chollet's book)]

4. Recurrent neural networks (RNNs). This part introduces RNNs and its applications in natural language processing (NLP).

   • Text processing. [slides]

   • RNN basics and LSTM. [slides] [reference]

   • Text generation. [slides]

   • Machine translation. [slides]

   • Image caption generation. [slides] [reference]

   • Attention. [slides] [reference-1] [reference-2]

5. Language Models beyond RNNs.

   • Transformer model: beyond RNNs. [slides] [reference]

   • Pre-train Transformer using BERT. [slides] [reference]

6. Autoencoders. This part introduces autoencoders for dimensionality reduction and image generation.

   • Autoencoder for dimensionality reduction. [slides]

   • Variational Autoencoders (VAEs) for image generation. [slides]

7. Generative Adversarial Networks (GANs).

   • DC-GAN [slides]

8. Recommender system. This part is focused on the collaborative filtering approach to recommendation based on the user-item rating data. This part covers matrix completion methods and neural network approaches.

   • Collaborative filtering. [slides]

9. Deep Reinforcement Learning.

   • Reinforcement learning [slides] [lecture note]

   • Value-based learning [slides]

   • Policy-based learning [slides]

   • Actor-critic methods [slides]

   • AlphaGo [slides]

10. Parallel Computing.

    • Basics and MapReduce. [slides] [lecture note] [Video (in Chinese)]

    • Parameter Server and Decentralized Network. [slides] [Video (in Chinese)]

    • Federated Learning. [slides] [Video (in Chinese)]

11. Adversarial Robustness. This part introduces how to attack neural networks using adversarial examples and how to defend from the attack.

    • Data evasion attack and defense. [slides] [lecture note]

    • Further reading: [Adversarial Robustness - Theory and Practice]