My Awesome Book

This file file serves as your book's preface, a great place to describe your book's content and ideas.

To have a basic mathematical background, you need to have some knowledge of the following mathematical concepts:

Probability and statistics
Linear algebra
Optimization
Multivariable calculus
Functional analysis (not essential)
First-order logic (not essential)
You can find some reasonable material on most of these by searching for "<topic> lecture notes" on Google. Usually, you'll find good lecture notes compiled by some professor teaching that course. The first few results should give you a good set to choose from. SeePrasoon Goyal's answer to How should I start learning the maths for machine learning and from where?
Skim through these. You don't need to go through them in a lot of detail. You can come back to studying the math as and when required while learning ML.

Then, for a quick overview of ML, you can follow the roadmap below (I’ve written about many of these topics in various answers on Quora; I linked the most relevant ones for quick reference.)

An Extensive Step by Step Guide to Exploratory Data Analysis: https://towardsdatascience.com/an-extensive-guide-to-exploratory-data-analysis-ddd99a03199e

Basic Statistics You NEED to Know for Data Science: https://towardsdatascience.com/basic-statistics-you-need-to-know-for-data-science-1fdd290f59b5

Day 1:

Basic terminology:
1. Most common settings: Supervised setting, Unsupervised setting, Semi-supervised setting, Reinforcement learning.
2. Most common problems: Classification (binary & multiclass), Regression, Clustering.
3. Preprocessing of data: Data normalization.
Concepts of hypothesis sets, empirical error, true error,
complexity of hypotheses sets, regularizatin, bias-variance trade-off, loss functions, cross-validation
.

Day 2:

Optimization
basics:
1. Terminology & Basic concepts: Convex optimization, Lagrangian, Primal-dual problems, Gradients & subgradients, ℓ1 and ℓ2 regularized objective functions
2. Algorithms: Batch gradient descent & stochastic gradient descent , Coordinate gradient descent.
3. Implementation: Write code for stochastic gradient descent for a simple objective function, tune the step size, and get an intuition of the algorithm.

Day 3:

Classification:
1. Logistic Regression
2. Support vector machines: Geometric intuition, primal-dual formulations, notion of support vectors, kernel trick, understanding of hyperparameters, grid search
3. Online tool for SVM : Play with this online SVM tool (scroll down to “Graphic Interface”) to get some intuition of the algorithm.

Day 4:

Regression:
1. Ridge regression
Clustering:
1. k-means & Expectation-Maximization algorithm.
2. Top-down and bottom-up hierarchical clustering.

Day 5:

Bayesian methods:
1. Basic terminology: Priors, posteriors, likelihood, maximum likelihood estimation and maximum-a-posteriori inference.
2. Gaussian Mixture Models
3. Latent Dirichlet Allocation : The generative model and basic idea of parameter estimation.

Day 6:

Graphical models:
1. Basic terminology: Bayesian networks, Markov networks / Markov random fields.
2. Inference algorithms: Variable elimination, Belief propagation.
3. Simple examples: Hidden Markov Models. Ising model.

Days 7–8:

Neural Networks
1. Basic terminology: Neuron, Activation function, Hidden layer.
2. Convolutional neural networks: Convolutional layer, pooling layer, Backpropagation.
3. Memory-based neural networks: Recurrent Neural Networks , Long-short term memory.
4. Tutorials: I’m familiar with this Torch tutorial (you’ll want to look at 𝟷_𝚜𝚞𝚙𝚎𝚛𝚟𝚒𝚜𝚎𝚍 1_supervised directory). There might be other tutorials in other deep learning frameworks.

Day 9:

Miscellaneous topics:
1. Decision trees
2. Recommender systems
3. Markov decision processes
4. Multi-armed bandits

Day 10:(Budget day)

You can use the last day to catch up on anything left from previous days, or learn more about whatever topic you found most interesting / useful for your future work.

Once you’ve gone through the above, you’ll want to start going through some standard online course or ML text. Andrew Ng's course on Coursera is a good starting point. An advanced version of the course is available on The Open Academy (Machine Learning | The Open Academy). The popular books that I have some experience with are the following:

While Murphy's book is more current and is more elaborate, I find Bishop’s to be more accessible for beginners. You can choose one of them according to your level.
At this point, you should have a working knowledge of machine learning. Beyond this, if you're interested in a particular topic, look for specific online resources on the topic, read seminal papers in the subfield, try finding some simpler problems and implement them.

It is important to implement some basic algorithms when you start doing ML, like gradient descent, AdaBoost, decision trees, etc [in addition to whatever you’ve implemented in the 10-day overview]. You should also have some experience with data preprocessing, normalization, etc. Once you have implemented a few algorithms from scratch, for other algorithms, you should use the standard implementations (like LibSVM, Weka, ScikitLearn, etc) on some toy problems, and get a good understanding of different algorithms.

binshi/ml-ds

My Awesome Book