/dl_tutorials_10weeks

Deep Learning Tutorials for 10 Weeks

MIT LicenseMIT

Deep Learning Tutorial

45 Papers + TF implementations

Topics (papers)

Modern CNNs

  • Alex Krizhevsky, et al. "ImageNet Classification with Deep Convolutional Neural Networks", NIPS, 2012
  • Christian Szegedy, et al. "Going Deeper with Convolutions", CVPR, 2015
  • Christian Szegedy, et al. "Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning", ArXiv, 2016
  • Kaiming He, et al. "Deep Residual Learning for Image Recognition", CVPR, 2016
  • Andreas Veit, et al. "Residual Networks are Exponential Ensembles of Relatively Shallow Networks", ArXiv, 2016
  • Sergey Zagoruyko and Nikos Komodakis "Wide Residual Networks", ArXiv, 2016

Regularization

  • Nitish Srivastava, et al. "Dropout- A Simple Way to Prevent Neural Networks from Overfitting", JMLR, 2014
  • Sergey Ioffe and Christian Szegedy "Batch Normalization- Accelerating Deep Network Training by Reducing Internal Covariate Shift, ArXiv, 2015

Algorithms behind AlphaGo

  • David Silver et al. "Mastering the game of Go with deep neural networks and tree search", Nature, 2016

Optimization Methods

  • Momentum, NAG, AdaGrad, AdaDelta, RMSprop, ADAM
  • Diederik Kingma and Jimmy Bam "ADAM: A Method For Stochastic Optimization", ICLR, 2015

Restricted Boltzmann Machine

  • Geoffrey Hinton, "A Practical Guide to Training Restricted Boltzmann Machines", 2010

Semantic Segmentation

  • Jonathan Long et al. "Fully Convolutional Networks for Semantic Segmentation", CVPR, 2015
  • Liang-Chieh Chen et al. "Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs", CVPR, 2015
  • Hyeonwoo Noh et al. "Learning Deconvolution Network for Semantic Segmentation", ICCV, 2015
  • Liang-Chieh Chen et al. "DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs", ArXiv, 2016

Weakly Supervised Localization

  • Maxime Oquab et al. "Is object localization for free? – Weakly-supervised learning with convolutional neural networks", CVPR, 2015
  • Bolei Zhou et al. "Learning Deep Features for Discriminative Localization", CVPR, 2016

Image detection methods

  • Ross Girshick et al. "Rich feature hierarchies for accurate object detection and semantic segmentation", CVPR, 2014
  • Kaiming He et al. "Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition", CVPR, 2015
  • Ross Girshick, "Fast R-CNN", ICCV, 2015
  • Shaoqing Ren et al. "Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks", NIPS, 2015
  • Joseph Redmon et al. "You Only Look Once: Unified, Real-Time Object Detection", CVPR, 2016
  • Donggeun Yoo et al. "AttentionNet: Aggregating Weak Directions for Accurate Object Detection", ICCV, 2015
  • Wei Liu et al. "SSD: Single Shot MultiBox Detector", ECCV, 2016
  • Joseph Redmon, Ali Farhadi, "YOLO9000: Better, Faster, Stronger", ArXiv, 2017

Visual Q&A

  • Hyeonwoo Noh et al. "Image Question Answering using Convolutional Neural Network with Dynamic Parameter Prediction", CVPR, 2015
  • Akira Fukui et al. "Multimodal Compact Bilinear Pooling for VQA", CVPR, 2016

Deep reinforcement learning

  • Volodymyr Mnih et al. "Playing Atari with Deep Reinforcement Learning", NIPS, 2013
  • Hado van Hasselt et al. "Deep Reinforcement Learning with Double Q-learning", AAAI, 2016

Recurrent Neural Networks

  • Alex Graves, "Generating Sequences With Recurrent Neural Networks", ArXiv, 2013

Word embedding

  • Tomas Mikolov et al. "Distributed Representations of Words and Phrases and their Compositionality", NIPS, 2013

Image captioning

  • Oriol Vinyals et al. "Show and Tell: A Neural Image Caption Generator", CVPR, 2015
  • Kelvin Xu et al. "Show, Attend and Tell: Neural Image Caption Generation with Visual Attention", ICML, 2015
  • Justin Johnson et al. "DenseCap: Fully Convolutional Localization Networks for Dense Captioning", CVPR, 2016

Neural Styles

  • Leon A. Gatys et al. "Texture Synthesis Using Convolutional Neural Networks", NIPS, 2015
  • Aravindh Mahendran and Andrea Vedaldi, "Understanding Deep Image Representations by Inverting Them", CVPR, 2015
  • Leon A. Gatys et al. "A Neural Algorithm of Artistic Style", ArXiv, 2015

Generative adversarial networks

  • Ian J. Goodfellow et al. "Generative Adversarial Networks", NIPS, 2015
  • Alec Radford et al. "Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks", ICLR, 2016
  • Scott Reed et al. "Generative Adversarial Text to Image Synthesis", ICML, 2016
  • Donggeun Yoo et al. "Pixel Level Domain Transfer", ECCV, 2016
  • Phillip Isola et al, "Image-to-Image Translation with Conditional Adversarial Networks", ArXiv, 2016
  • Anh Nguyen et al. "Plug & Play Generative Networks: Conditional Iterative Generation of Images in Latent Space", ArXiv, 2016
  • Scott Reed et al. "Learning What and Where to Draw", NIPS, 2016

and implementations (which can be found in TF-101)

  • Basic Python usage (numpy, matplotlib, ..)
  • Handling MNIST
  • Logistic regression
  • Multilayer Perceptron
  • Convolutional Neural Network
  • Denoising Autoencoders (+Convolutional)
  • Class Activation Map
  • Semantic Segmentation
  • Using Custom Dataset
  • Recurrent Neural Network
  • Char-RNN
  • Word2Vec
  • Neural Style