/dl_tutorials_4th

MIT LicenseMIT

Deep Learning Tutorial

Topics (papers)

Modern CNNs

  • ImageNet Classification with Deep Convolutional Neural Networks (AlexNet)
  • Going Deeper with Convolutions (GoogLeNet)
  • Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning
  • Deep Residual Learning for Image Recognition
  • Residual Networks are Exponential Ensembles of Relatively Shallow Networks
  • Wide Residual Networks

Regularization

  • Dropout- A Simple Way to Prevent Neural Networks from Overfitting
  • Batch Normalization- Accelerating Deep Network Training by Reducing Internal Covariate Shift

Algorithms behind AlphaGo

  • Mastering the game of Go with deep neural networks and tree search (AlphaGo)

Optimization Methods

  • Momentum, NAG, AdaGrad, AdaDelta, RMSprop, AdaM
  • ADAM: A Method For Stochastic Optimization

Restricted Boltzmann Machine

  • A Practical Guide to Training Restricted Boltzmann Machines (RBM)

Semantic Segmentation

  • Fully Convolutional Networks for Semantic Segmentation
  • Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs
  • Learning Deconvolution Network for Semantic Segmentation
  • DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs

Weakly Supervised Localization

  • Learning Deep Features for Discriminative Localization
  • Is object localization for free? – Weakly-supervised learning with convolutional neural networks

Image detection methods

  • Rich feature hierarchies for accurate object detection and semantic segmentation
  • Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition
  • Fast R-CNN
  • Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
  • You Only Look Once: Unified, Real-Time Object Detection
  • AttentionNet: Aggregating Weak Directions for Accurate Object Detection
  • SSD: Single Shot MultiBox Detector

Visual Q&A

  • Image Question Answering using Convolutional Neural Network with Dynamic Parameter Prediction
  • Multimodal Compact Bilinear Pooling for VQA

Deep reinforcement learning

  • Playing Atari with Deep Reinforcement Learning
  • Deep Reinforcement Learning with Double Q-learning

Recurrent Neural Networks

  • Generating Sequences With Recurrent Neural Networks

Word embedding

  • Distributed Representations of Words and Phrases and their Compositionality

Image captioning

  • Show and Tell: A Neural Image Caption Generator
  • Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
  • DenseCap: Fully Convolutional Localization Networks for Dense Captioning

Neural Styles

  • Texture Synthesis Using Convolutional Neural Networks
  • Understanding Deep Image Representations by Inverting Them
  • A Neural Algorithm of Artistic Style

Generative adversarial networks

  • Generative Adversarial Networks
  • Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
  • Generative Adversarial Text to Image Synthesis
  • Pixel Level Domain Transfer

and implementations

  • Basic Python usage (numpy, matplotlib, ..)
  • Handling MNIST
  • Logistic regression
  • Multilayer Perceptron
  • Convolutional Neural Network
  • Denoising Autoencoders (+Convolutional)
  • Class Activation Map
  • Semantic Segmentation
  • Using Custom Dataset
  • Recurrent Neural Network
  • Char-RNN
  • Word2Vec
  • Neural Style