/t81_558_deep_learning

WUSTL Course T81-558: Applications of Deep Neural Networks

Primary LanguageJupyter NotebookApache License 2.0Apache-2.0

T81 558:Applications of Deep Neural Networks

Washington University in St. Louis

Instructor: Jeff Heaton

Fall 2016, Mondays, 6 - 8:30 PM, Seigle, #109

Course Description

Deep learning is a group of exciting new technologies for neural networks. By using a combination of advanced training techniques neural network architectural components, it is now possible to train neural networks of much greater complexity. This course will introduce the student to deep belief neural networks, regularization units (ReLU), convolution neural networks and recurrent neural networks. High performance computing (HPC) aspects will demonstrate how deep learning can be leveraged both on graphical processing units (GPUs), as well as grids. Deep learning allows a model to learn hierarchies of information in a way that is similar to the function of the human brain. Focus will be primarily upon the application of deep learning, with some introduction to the mathematical foundations of deep learning. Students will use the Python programming language to architect a deep learning model for several of real-world data sets and interpret the results of these networks.

Objectives

  1. Explain how neural networks (deep and otherwise) compare to other machine learning models.
  2. Determine when a deep neural network would be a good choice for a particular problem.
  3. Demonstrate your understanding of the material through a final project uploaded to GitHub.

Syllabus

This syllabus presents the expected class schedule, due dates, and reading assignments. Download current syllabus.

Class Content
Class 1
08/29/2016
  • Python for Machine Learning
Labor Day
09/05/2016		 No class session
Class 2
09/12/2016
  • Neural Network & Machine Learning Basics
  • Introduction to TensorFlow
  • Assignment: Read Chapter 1
Class 3
09/19/2016
  • Training a Neural Network
  • Assignment: Read Chapters 4 & 5
Class 4
09/26/2016
  • Classification & Regression
  • Assignment: Program 1 Due
Class 5
10/03/2016
  • Backpropagation
  • Assignment: Read Chapter 6
Class 6
10/10/2016
  • Preprocessing
  • Assignment: Program 2 Due
Fall Break
10/17/2016		 No class session
NO CLASS
10/24/2016		 No class session
Class 7
10/31/2016
  • Convolutional Neural Networks
  • Assignment: Take Home Midterm Due
Class 8
11/07/2016
  • Kaggle Datasets
  • Evaluating Neural Network Performance
Class 9
11/14/2016
  • Regularization and Dropout
  • Assignment: Read Chapter 12, Program 3 Due
Class 10
11/21/2016
  • Timeseries and Recurrent Neural Networks
  • Assignment: Read Chapter 13
Class 11
11/28/2016
  • Architecting Neural Networks
  • Assignment: Read Chapter 14
Class 12
12/05/2016
  • Applications of Neural Networks
  • Assignment: Program 4 Due (submitted on Kaggle.com)
Class 13
12/12/2016
  • Neural Network Ensembles
Class 14
12/19/2016
  • GPU, HPC and Cloud
  • Assignment: Final Project Due

Datasets

  • Iris - Classify between 3 iris species.
  • Auto MPG - Regression to determine MPG.
  • WC Breast Cancer - Binary classification: malignant or benign.
  • toy1 - The toy1 dataset, regression for weights of geometric solids.

*Note: Other datasets will be added as the class progresses.

Assignments

Other Information