Visual representation learning Tutorials

Leren & Beslissen 2021

This repository serves as an introduction to deep learning for UvA students Leren & Beslissen course 2021 for the visual representation learning project. The repository consist of tutorial notebooks on deep learning and pytorch, autoencoders and Convolutional Neural Networks.

Tutorials

Under tutorials you can find notebooks on the following topics.

Part 1: Deep Learning & Pytorch
- Tensors and Neural Networks
- Training and backpropagation
- Validation and Inference
- Loading images and Transfer learning
Part 2: ConvNets & Visualization
- Convolutional Neural Networks on CIFAR-10
- Convolutional Visualization and Filters
Part 3: Autoencoders
- Introduction to Autoencoders on MNIST: Simple Autoencoder, Convolutional Autoencoder and Denoising Autoencoder

Resources

Here you can find some resources that might be useful to you, in addition to the references found in the project description.

Consider using Google Colab if you need access to more computing power. To learn more, see the Colab intro.
Additional resources on CNNs:
Additional resources on representation learning and Autoencoders:
- Autoencoder Wikipedia
- Representation learning blog
Additional resources on self-supervised learning (Note: might not be beginner-friendly)
- Self-supervised learning simple introduction
- Self-supervised learning blog

Getting Started

We recommend to configure and manage an environment with Anaconda, a package and environment manager.

Overview

Using Anaconda consists of the following:

Install miniconda on your computer, by selecting the latest Python version for your operating system. If you already have conda or miniconda installed, you should be able to skip this step and move on to step 2.
Create and activate * a new conda environment.

1. Installation

Download the latest version of miniconda that matches your system.

	Linux	Mac	Windows
64-bit	64-bit (bash installer)	64-bit (bash installer)	64-bit (exe installer)
32-bit	32-bit (bash installer)		32-bit (exe installer)

Install miniconda on your machine. Detailed instructions:

2. Create and Activate the Environment

For Windows users, these following commands need to be executed from the Anaconda prompt as opposed to a Windows terminal window. For Mac, a normal terminal window will work.

Git and version control

These instructions also assume you have git installed for working with Github from a terminal window, but if you do not, you can download that first with the command:

conda install git

Now, we're ready to create our local environment!

Clone the repository, and navigate to the downloaded folder. This may take a minute or two to clone due to the included image data.

git clone https://github.com/martinetoering/visual-representation-learning-tutorials

Create (and activate) a new environment, named representation-learning with Python 3.6. If prompted to proceed with the install (Proceed [y]/n) type y.
- Linux or Mac:
```
conda create -n representation-learning python=3.6
conda activate representation-learning
```
- Windows:
```
conda create --name representation-learning python=3.6
activate representation-learning
```
At this point your command line should look something like: (representation-learning) [user]. The (representation-learning) indicates that your environment has been activated, and you can proceed with further package installations.

Install PyTorch and torchvision; this should install the latest version of PyTorch.

Linux or Mac:

conda install pytorch torchvision -c pytorch

Windows:

conda install pytorch -c pytorch
pip install torchvision

Install a few required pip packages, which are specified in the requirements text file (including OpenCV).

pip install -r requirements.txt

Acknowledgement - References

The majority of the tutorial notebooks are from the lab assignments of the Udacity's Deep Learning Nanodegree program
Other materials come from the Stanford CS231n course

martinetoering/Visual-Representation-Learning-Tutorials