A collection of information sources on deep learning with the focus on practical understanding of the technology and its usage. The challenge I try to face is to gain the necessary depth of understanding on the topic to be able to generalize proven solutions without going as deep as active researcher on deep learning need to be. The collection has a strong bias to supervised learning.
This collection is maintained by Michael Krämer (@mkraemerx). If you think you can contribute to this list, please submit a pull request.
- Basics
- Data Exploration & Preprocessing
- Problem & Type Definition
- Model Architecture
- Training & Model Validation
- Practical Deep Learning For Coders: A free online course that is meant to take 7 weeks to get you into practical usage of deep learning. The creators state that deep learning needs to get out of the ivory tower and present a course that requires almost no math.
- Visual Information Theory
- How Convolutional Neural Networks see the world: Understanding of what is happening inside of a deep learning network is quite a challenge. This blog describes a visual approach to understand the network and its layers.
- Matrices
- Properties of Functions:
- Continuous
- Monotonic Function
- Differentiable
- Non-negative
- ...
- Universal Approximator Theorem (1993)
- No Free Lunch Theorems by David Wolpert, William Macready
- Introduction to Data Exploration: Gives a good first overview about data exploration and cleanup. More targeted to classical machine learning, especially the sections about outliers and feature engineering should partly not be necessary in deep learning.
- K-Means in SciKit-Learn
- Dataset splitting in SciKit-Learn
- Google Cloud Dataprep: Tool in beta program that guesses data types and creates basic uni-variate properties and histograms. Interactive. Seems it has been evolved out of DataWrangler.
- Normalization in DeepLearning4J
-
Encoding categorical features: Also know as One-hot-vectors or One-hot-encoding, this type of encoding performs way better than usage of ordinal types (numbers) to represent categories.
-
Lower bound of needed observations for sparse, high-dimensional feature spaces is k for 2^k regions: Non-Local Manifold Parzen Windows by Bengio Y., Larochelle H and Vincent P (in NIPS‘2005, MIT Press)
- OpenCV: Targeted to computer vision, ships with flexible recipes.
- NLP_BestPractices: Best practices for text processing.
- Keras: Meta-API for definition of deep neuronal networks.
- Estimator in Tensorflow: High-level API for Tensorflow.
- LossFunction: The function measure of the preciseness of predictions from the model.
- Dataset Augmentation