I am implementing from scratch the tools, techniques and best practices I learnt from fast.ai's 2017 offering.
The idea is to not use the fastai library and build everything minimally on PyTorch, to see things happening first hand.
I have referred code from the following sources (the exact source is also mentioned in each file):
Code tested on PyTorch v0.2.0, Python 3.6
- Load from folders segregated into classes
- Load from csv files assigning classes to images
- Horizontal flip
- Cropping center
- Cropping random
- Cropping custom
- Scalling
- Optimum learning rate finder
- LR Annealing
- Using pretrained weights
- Freezing layers
- Precompute activations
- Delete/Add Layers
- Test Time Augmentation (TTA)
These papers or reading material was suggested either by Jeremy or other participants in various discussions.
- GDR: Stochastic Gradient Descent with Warm Restarts
- Cyclical Learning Rates for Training Neural Networks
- Overcoming catastrophic forgetting in neural networks
- Don't Decay the Learning Rate, Increase the Batch Size
- Using convolutional networks and satellite imagery to identify patterns in urban environments at a large scale