/stat212b

Topics Course on Deep Learning UC Berkeley

stat212b

Topics Course on Deep Learning for Spring 2016

by Joan Bruna, UC Berkeley, Statistics Department

##Syllabus

1st part: Convolutional Neural Networks

  • Invariance, stability.
  • Variability models (deformation model, stochastic model).
  • Scattering
  • Extensions
  • Group Formalism
  • Supervised Learning: classification.
  • Properties of CNN representations: invertibility, stability, invariance.
  • covariance/invariance: capsules and related models.
  • Connections with other models: dictionary learning, LISTA, Random Forests.
  • Other tasks: localization, regression.
  • Embeddings (DrLim), inverse problems
  • Extensions to non-euclidean domains.
  • Dynamical systems: RNNs and optimal control.
  • Guest Lecture: Wojciech Zaremba (OpenAI)

2nd part: Deep Unsupervised Learning

  • Autoencoders (standard, denoising, contractive, etc.)
  • Variational Autoencoders
  • Adversarial Generative Networks
  • Maximum Entropy Distributions
  • Open Problems
  • Guest Lecture: Soumith Chintala (Facebook AI Research)

3rd part: Miscellaneous Topics

  • Non-convex optimization theory for deep networks
  • Stochastic Optimization
  • Attention and Memory Models
  • Guest Lecture: Yann Dauphin (Facebook AI Research)

Schedule