Jeremy Howard and Rachhel Thomas teaches practical deep learning at fast.ai. The course aims a practical learning process rather than covering theoretical concepts from ground-up. Therefore includes lots of practical tips and i think this is highly useful for anybody who wants to dive into details (even if ones with the knowledge of linear algebra, statistics, machine learning, some deep learning, etc).
This lesson includes tools and tricks for machine learning people. Dives into the kaggle competition named cats and dogs redux in order to cover classification and transfer learning (by fine tuning the imagenet model vgg16). Jeremy also tells about making deep learning inclusive by making neural networks uncool again! In conclusion, we are in the club, thank you for that!
solutions for lesson 01 are here and here.
Lesson 2, covers convolutional neural networks and visualization of convolutional layers to show why transfer learning is applicable. solutions are here.
Papers suggested in the lession plus additional ones.
- M. D. Zeiler, R. Fergus, "Visualizing and Understanding Convolutional Networks", arXiv, 2013.
- Jason Yosinski, Jeff Clune, Yoshua Bengio, Hod Lipson, "How transferable are features in deep neural networks?", arXiv, 2014.
This lesson covers the details of overfitting and underfitting, dropout is not used to avoid the underfitting and, after that, data augmentation techniques are applied to avoid overfitting. Batch normalization is covered.
I have implemented the bottleneck feature saving process here, However it is only a demonstration with the sample set. Bottleneck features for the train set is about 2GB in size, even with 8gb GPUs of amazon i wasing able to fit the feature arrays in memory to save. Now i am using 1080 ti but i skipped the feature saving step and fine tuned the dense layers by just freezing the conv layers which is implemented here. And batch normalization is here.