/Practical_DL

DL course co-developed by HSE, YSDA and Skoltech

Primary LanguageJupyter NotebookMIT LicenseMIT

Deep learning course

This repo supplements Deep Learning course taught at YSDA and Skoltech @spring'19. For previous iteration visit the fall18 branch.

Lecture and seminar materials for each week are in ./week* folders. Homeworks are in ./homework* folders.

General info

  • Create cloud jupyter session from this repo - Binder
  • Telegram chat room (russian).
  • YSDA deadlines & admin stuff can be found at the YSDA course wiki (ysda students only).
  • Any technical issues, ideas, bugs in course materials, contribution ideas - add an issue

Syllabus

  • week01 Intro to deep learning

    • Lecture: Deep learning -- introduction, backpropagation algorithm
    • Seminar: Neural networks in numpy
    • Homework 1 is out!
    • Please begin worrying about installing pytorch. You will need it next week!

  • week02 Adaptive optimization methods

    • Lecture: Empirical risk minimization, standard loss functions, linear classification, stochastic optimizers, adaptive SGD
    • Seminar: PyTorch basics

  • week03 Convolutional networks I

    • Lecture: Convolutional networks (ConvNets), computer vision
    • Seminar: Convolutional networks in pytorch
    • Homework 2 is out!

  • week04 Convolutional networks II

    • Lecture: ConvNet architectures, representations inside CNNs; visualizing networks/inceptionism, transfer learning
    • Seminar: Fine-tuning a pre-trained network

  • week05 Advanced Computer vision

    • Lecture: "Deep" computer vision beyond classification; Verification tasks, object detection architectures, semantic segmentation
    • Seminar: Semantic segmentation
    • Homework 3 is out!

  • week06 Deep generative models I

    • Lecture: Deep image generation; generative ConvNets, perceptual loss functions.
    • Seminar: Art Style Transfer by Dmitry Ulyanov

  • week07 Deep generative models II

    • Lecture: Generative Adversarial Networks
    • Seminar: Generative Adversarial Networks

  • week08 Unsupervised deep learning

    • Lecture: Autoencoders, variational autoencoders, image analogies
    • Seminar: Variational autoencoders

  • week09 Deep learning for natural language processing

    • Lecture: Word embeddings, word2vec and other variants, convolutional networks for natural language
    • Seminar: Word embeddings. Text convolutions for salary prediction.
    • Homework 4 is out!

  • week10 Recurrent neural networks

    • Lecture: Modelling sequences. Simple RNN. Why BPTT isn't worth 4 letters. GRU/LSTM.
    • Seminar: Generating human names and deep learning papers with RNNs

  • week11 Recurrent neural networks II

    • Lecture: Sequence2sequence, architectures with attention and long-term memory.
    • Seminar: Image Captioning

  • week12: Deep Reinforcement Learning

    • Lecture: Reinforcement Learning, MDPs, policy gradient methods
    • Seminar: REINFORCE on simple robot control, optional: advantage actor-critic on atari

Contributors & course staff

Course materials and teaching performed by