xiaoerlaigeid/machine-learning-notes

This contains my machine learning notes in latex form

Video Tutorial to these notes.

• I recorded about 20% of these notes in videos in 2015 in Mandarin (all my notes and writings are in English) You may find them on Youtube and 优酷

• I always look for high quality PhD students in Machine Learning, both in terms of probabilistic model and Deep Learning models. Contact me on YiDa.Xu@uts.edu.au

Data Science

• 30 minutes introduction to AI and Machine Learning

An extremely gentle 30 minutes introduction to AI and Machine Learning. Thanks to my PhD student Haodong Chang for assist editing

• Regression methods

Classification: Logistic and Softmax; Regression: Linear, polynomial; Mix Effect model

• Recommendation system

collaborative filtering, Factorization Machines, Non-Negative Matrix factorisation, Multiplicative Update Rule

• Dimension Reduction

classic PCA and t-SNE

• Introduction to Data Analytics and associate Jupyter notebook

Three perspectives into machine learning and Data Science. Supervised vs Unsupervised Learning, Classification accuracy

Deep Learning (jupyter style notes coming in 2018)

• Optimisation methods

Optimisation methods in general. not limited to just Deep Learning

• Neural Networks

basic neural networks and multilayer perceptron

• Convolution Neural Networks: from basic to recent Research

detailed explanation of CNN, various Loss function, Centre Loss, contrastive Loss, Residual Networks, YOLO, SSD

• Other Deep Learning topics

Other Deep learning models including RNN, GAN and RBM

• Deep Reinforcement Learning

basic knowledge in reinforcement learning, Markov Decision Process, Bellman Equation and move onto Deep Q-Learning

Probability and Statistics Background

• Bayesian model

revision on Bayes model include Bayesian predictive model, conditional expectation

• Probabilistic Estimation

some useful distributions, conjugacy, MLE, MAP, Exponential family and natural parameters

• Statistics Properties

useful statistical properties to help us prove things, include Chebyshev and Markov inequality

Probabilistic Model

• Expectation Maximisation

Proof of convergence for E-M, examples of E-M through Gaussian Mixture Model

• State Space Model (Dynamic model)

explain in detail of Kalman Filter and Hidden Markov Model

Inference

• Variational Inference

explain Variational Bayes both the non-exponential and exponential family distribution plus stochastic variational inference.

• Stochastic Matrices

stochastic matrix, Power Method Convergence Theorem, detailed balance and PageRank algorithm

• Introduction to Monte Carlo

inverse CDF, rejection, adaptive rejection, importance sampling

• Markov Chain Monte Carlo

M-H, Gibbs, Slice Sampling, Elliptical Slice sampling, Swendesen-Wang, demonstrate collapsed Gibbs using LDA

• Particle Filter (Sequential Monte-Carlo)

Sequential Monte-Carlo, Condensational Filter algorithm, Auxiliary Particle Filter

Advanced Probabilistic Model

• Bayesian Non Parametrics (BNP) and its inference

Dircihlet Process (DP), Hierarchical DP, HDP-HMM, Slice sampling for DP

• Determinantal Point Process

explain the details of DPP’s marginal distribution, L-ensemble, its sampling strategy, our work in time-varying DPP