ocordeiro/eec2003

Repository for EEC2003, a graduate course on PPgEEC about Fundamentals of Deep Learning

Universidade Federal do Rio Grande do Norte

Programa de Pós-Graduação em Engenharia Elétrica e de Computação PPgEEC

EEC2003 - Special Topics in AI

• Lesson #01 Introduction
  • Course Outline & Presentation
  • Google Colab Introduction
  • Google Colab Cont.
• Lesson #02 Fundamentals
  • Introduction to Deep Learning and TensorFlow - Outline
  • The perceptron
  • Building Neural Networks
  • Matrix Dimension
  • Applying Neural Networks
  • Training a Neural Network
  • Backpropagation with Pencil & Paper
  • Learning rate & Batch Size
  • Exponentially Weighted Average
  • Adam, Momentum, RMSProp, Learning Rate Decay
• Lesson #03 Better Generalization vs Better Learning
  • Better Generalization
    • Better Deep Learning - Outline
    • Spliting Data
    • Bias vs Variance
    • Weight Regularization
    • Weight Constraint
    • Dropout
    • Promote Robustness with Noise
    • Early Stopping
  • Better Learning
    • Data scaling
    • Vanishing/Exploding Gradient
    • Fix Vanishing Gradient with Relu
    • Fix Exploding Gradient with Gradient Clipping
• Lesson #04 Hyperparameter Tuning & Batch Normalization
  • Outline
  • Hyperparameter Tuning Fundamentals
  • Keras Tuner and Weights and Biases
  • Wandb - Part #01
  • Wandb - Part #02
  • Batch Normalization Fundamentals
  • Batch Normalization Math Details
  • Batch Normalization Case Study
• Lesson #05 Fundamentals of Convolutional Neural Networks (CNN)
  • Outline
  • CNN Introduction & Motivation
  • Convolutional Layer
  • Case Study of Convolutional Layer
  • Pooling Layer
  • Fully Connected Layer
  • Case Study - Signs Dataset
• Lesson #06 Convolutional Neural Networks (CNN) Architecture I
  • Outline
  • Typical CNN Architecture
  • Best practices when building your own CNN
  • LeNet-5
  • Training LeNet-5 using MNIST dataset
  • ImageNet & ILSVRC
  • AlexNet
  • Dogs vs Cats Challenge + HDF5
  • CNN Architectures - Hands on Part #01
  • CNN Architectures - Hands on Part #02
• Lesson #07 Convolutional Neural Networks (CNN) Architecture II
  • Outline
  • VGG
  • VGG case study using CIFAR-10
  • How to use 1x1 convolutions
  • GoogLeNet
  • MiniGoogleLeNet using CIFAR-10
  • DeeperGoogLeNet and Tiny ImageNet Challenge
• Lesson #08 ResNet
• Lesson #09 Transfer Learning
  • Outline
  • Feature Extractors
  • Feature Extractors - Case Study
  • Fine-Tuning
  • Fine-Tuning Case Study I - Flowers 17
  • Fine-Tuning Case Study II - Cats and Dogs