This repository contains my exercises and projects for the Deep Learning Nanodegree at Udacity. You can find the original exercise and project notebooks for this course at the deep-learning-v2-pytorch and sagemaker-deployment repositories.
Predicting Bike-Sharing Patterns Project
In this project, I build a neural network from scratch to carry out a prediction problem on a real dataset! By building a neural network from the ground up, I have a much better understanding of gradient descent, backpropagation, and other concepts that are important to know before moving on to higher level tools such as PyTorch. The data comes from the UCI Machine Learning Database.
In this project, I build a pipeline to process real-world, user-supplied images. Given an image of a dog, my algorithm will identify an estimate of the canine’s breed by using state-of-the-art CNN models for classification. If supplied an image of a human face, the code will identify the resembling dog breed.
In this project, I generate my own Seinfeld TV scripts using a recurrent neural network (RNN). The dataset used to train the network consists of scripts from 9 Seinfeld seasons. After training on the dataset, the RNN can create a new, "fake" TV script.
A script generated by my RNN:
kramer: listen jerry, i'm sorry. i just don't know what it is.
jerry: what?
george: you don't know? how could i go out with her?
jerry: no.
helen: what are you doing?
jerry: what about that?
george:(to the phone) hey. i don't know what you're saying.
jerry: i don't think so.
george: yeah...
jerry: oh, that's right.
george: yeah, right. i was wondering if you could do your own business. it's not that.
jerry: i don't have you.
kramer: no, no, don't you?... no, i want you to be there and i can have the car.
george: oh, oh, i can't believe this isn't a good time.
jerry: what is this?
george:(to the man) oh, i gotta go!
jerry: i got a lot of people in the car.
kramer: yeah.
jerry: hey.
kramer: yeah, you got it, i just want to be a good one.
kramer: yeah, yeah.(she looks at her watch) oh, you know, you got my keys, you were right on the top.
kramer:(to kramer) hey!
kramer: hey, hey jerry.
elaine:(to kramer) hey you guys, what are you doing here?
kramer: i didn't know i said" hello"
jerry:"(to kramer) so, what do you have to say?
jerry: i can't.
george: i just think i'm gonna go to the bathroom and get the rest of my life. i have to do this.
jerry: what are you talking about?
george: why did you get that?
jerry: well, it's like an accident.
jerry: what did you say?
george:(laughing) you got that
In this project, I define and train deep convolutional generative adversarial networks (DCGAN) on a dataset of faces. The goal is to get a generator network to generate new images of faces that look as realistic as possible. The CelebFaces Attributes Dataset (CelebA) was used to train the adversarial networks.
Some faces generated by my DCGAN:
Sentiment Analysis Web App Project
In this project, I build and deploy a neural network which predicts the sentiment of a user-provided movie review. In addition, I create a simple web app that uses my deployed model.
- Introduction to Neural Networks - In this exercise, I implement gradient descent and backpropagation in python and built a 2-layer Neural Network.
- Sentiment Analysis Network - In this exercise, Andrew Trask, the author of Grokking Deep Learning, walk me through using neural networks for sentiment analysis.
- Deep Learning with PyTorch - In this exercise, I learn how to use PyTorch for building deep learning models.
- Convolutional Neural Networks - In this exercise, I implement CNNs for classification tasks and learn how to visualize the inside of networks.
- Transfer Learning - In this exercise, I learn how to apply a pre-trained network to a new problem with transfer learning.
- Weight Initialization - In this exercise, I learn how to find good initial weights for a neural network to make it train better.
- Autoencoder - In this exercise, I build autoencoders using PyTorch.
- Style Transfer - In this exercise, I learn how to use a pre-trained network to extract content and style features from an image, and transfer it to another.
- Implementation of RNN & LSTM - In this exercise, I define and train RNNs in PyTorch and apply them to tasks that involve sequential data. Implemented Character-Level LSTM in PyTorch.
- Embeddings & Word2Vec - In this exercise, I learn about embeddings in neural networks by implementing the Word2Vec model.
- Sentiment Prediction RNN - In this exercise, I implement a sentiment prediction RNN for predicting whether a movie review is positive or negative.
- MNIST GAN - In this exercise, I learn to build a generative adversarial network (GAN) trained on the MNIST dataset. From this, I am able to generate new handwritten digits.
- Batch Normalization - This exercise show me how a simple MNIST classification model improves with the addition of batch normalization layers.
- DCGAN, SVHN - In this exercise, I learn to build a Deep Convolutional GAN (DCGAN) that can generate new images that look like house addresses from the Google Streetview dataset, SVHN.
- Implementing a CycleGAN - In this exercise, I implement a CycleGAN in PyTorch that can translate images from the summer to winter domains.
- Boston Housing (Batch Transform) - High Level is the simplest notebook which introduces you to the SageMaker ecosystem and how everything works together. The data used is already clean and tabular so that no additional processing needs to be done. Uses the Batch Transform method to test the fit model.
- Boston Housing (Batch Transform) - Low Level performs the same analysis as the low level notebook, instead using the low level api. As a result it is a little more verbose, however, it has the advantage of being more flexible. It is a good idea to know each of the methods even if you only use one of them.
- Boston Housing (Deploy) - High Level is a variation on the Batch Transform notebook of the same name. Instead of using Batch Transform to test the model, it deploys and then sends the test data to the deployed endpoint.
- Boston Housing (Deploy) - Low Level is again a variant of the Batch Transform notebook above. This time using the low level api and again deploys the model and sends the test data to it rather than using the batch transform method.
- IMDB Sentiment Analysis - XGBoost - Web App creates a sentiment analysis model using XGBoost and deploys the model to an endpoint. Then describes how to set up AWS Lambda and API Gateway to create a simple web app that interacts with the deployed endpoint.
- Boston Housing (Hyperparameter Tuning) - High Level is an extension of the Boston Housing XGBoost model where instead of training a single model, the hyperparameter tuning functionality of SageMaker is used to train a number of different models, ultimately using the best performing model.
- Boston Housing (Hyperparameter Tuning) - Low Level is a variation of the high level hyperparameter tuning notebook, this time using the low level api to create each of the objects involved in constructing a hyperparameter tuning job.
- Boston Housing - Updating an Endpoint is another extension of the Boston Housing XGBoost model where in addition we construct a Linear model and switch a deployed endpoint between the two constructed models. In addition, we look at creating an endpoint which simulates performing an A/B test by sending some portion of the incoming inference requests to the XGBoost model and the rest to the Linear model.
- IMDB Sentiment Analysis - XGBoost (Batch Transform) is a notebook that is to be completed which leads you through the steps of constructing a model using XGBoost to perform sentiment analysis on the IMDB dataset.
- IMDB Sentiment Analysis - XGBoost (Hyperparameter Tuning) is a notebook that is to be completed and which leads you through the steps of constructing a sentiment analysis model using XGBoost and using SageMaker's hyperparameter tuning functionality to test a number of different hyperparameters.
- IMDB Sentiment Analysis - XGBoost (Updating a Model) is a notebook that is to be completed and which leads you through the steps of constructing a sentiment analysis model using XGBoost and then exploring what happens if something changes in the underlying distribution. After exploring a change in data over time you will construct an updated model and then update a deployed endpoint so that it makes use of the new model.