spatre/machine-learning-notes

My continuously updated Machine Learning, Probabilistic Models and Deep Learning notes and demos (1000+ slides) 我不间断更新的机器学习，概率模型和深度学习的讲义(1000+页)和视频链接

Recent Research Talks

• When Probabilities meet Neural Networks (PPT) Presentation at Sinovation DeeCamp & Peking University, July 2018

Topics include: Expectation-Maximization & Matrix Capsule Networks; Determinantal Point Process & Neural Networks compression; Kalman Filter & LSTM; Model estimation & Binary classifier

• Noise Contrastive Estimation, Re-parameterization, Natural Gradients

Detailed illustration of Noise Contrastive Estimation (detals & derivations), Probability Density Re-parameterization, Natural Gradients

Video Tutorial to these notes

• I recorded about 20% of these notes in videos in 2015 in Mandarin (all my notes and writings are in English) You may find them on Youtube and bilibili and Youku

Data Science

• 30 minutes introduction to AI and Machine Learning

An extremely gentle 30 minutes introduction to AI and Machine Learning. Thanks to my PhD student Haodong Chang for assist editing

• Regression methods

Classification: Logistic and Softmax; Regression: Linear, polynomial; Mix Effect model [costFunction.m] and [soft_max.m]

[costFunction.m] [soft_max.m]

• Recommendation system

collaborative filtering, Factorization Machines, Non-Negative Matrix factorisation, Multiplicative Update Rule

• Dimension Reduction

classic PCA and t-SNE

• Introduction to Data Analytics and associate Jupyter notebook

Supervised vs Unsupervised Learning, Classification accuracy

Deep Learning

• Optimisation methods

Optimisation methods in general. not limited to just Deep Learning

• Neural Networks

basic neural networks and multilayer perceptron

• Convolution Neural Networks: from basic to recent Research

detailed explanation of CNN, various Loss function, Centre Loss, contrastive Loss, Residual Networks, Capsule Networks, YOLO, SSD YOLO，SSD

• Word Embeddings

Word2Vec, skip-gram, GloVe, Fasttext

• Deep Natural Language Processing

RNN, LSTM, Seq2Seq with Attenion, Beam search, Attention is all you need, Convolution Seq2Seq, Pointer Networks

"Attention is all you need",Seq2Seq

• Deep Reinforcement Learning

basic knowledge in reinforcement learning, Markov Decision Process, Bellman Equation and move onto Deep Q-Learning (under construction)

• Restricted Boltzmann Machine

basic knowledge in Restricted Boltzmann Machine (RBM)

Probability and Statistics Background

• Bayesian model

revision on Bayes model include Bayesian predictive model, conditional expectation

• Probabilistic Estimation

some useful distributions, conjugacy, MLE, MAP, Exponential family and natural parameters

• Statistics Properties

useful statistical properties to help us prove things, include Chebyshev and Markov inequality

Probabilistic Model

• Expectation Maximisation

Proof of convergence for E-M, examples of E-M through Gaussian Mixture Model, [gmm_demo.m] and [kmeans_demo.m] and [Youku]

[gmm_demo.m] 和 [kmeans_demo.m] (http://v.youku.com/v_show/id_XMTM1MjY1MDU5Mg)**

• State Space Model (Dynamic model)

explain in detail of Kalman Filter [Youku], [kalman_demo.m] and Hidden Markov Model [Youku]

(http://v.youku.com/v_show/id_XMTM1MzQ1NDk5Ng), [kalman_demo.m] (http://v.youku.com/v_show/id_XMTM2ODU1MzMzMg)**

Inference

• Variational Inference

explain Variational Bayes both the non-exponential and exponential family distribution plus stochastic variational inference. [vb_normal_gamma.m] and (http://v.youku.com/v_show/id_XMTM1Njc5NzkxNg)**

[vb_normal_gamma.m] (http://v.youku.com/v_show/id_XMTM1Njc5NzkxNg)**

• Stochastic Matrices

stochastic matrix, Power Method Convergence Theorem, detailed balance and PageRank algorithm

• Introduction to Monte Carlo

inverse CDF, rejection, adaptive rejection, importance sampling [adaptive_rejection_sampling.m] and [hybrid_gmm.m]

[adaptive_rejection_sampling.m] 和 [hybrid_gmm.m]

• Markov Chain Monte Carlo

M-H, Gibbs, Slice Sampling, Elliptical Slice sampling, Swendesen-Wang, demonstrate collapsed Gibbs using LDA [lda_gibbs_example.m] and [test_autocorrelation.m] and [gibbs.m] and [Youku]

[lda_gibbs_example.m] 和 [test_autocorrelation.m] 和 [gibbs.m] (http://v.youku.com/v_show/id_XMTM1NjAyNDYyNA)**

• Particle Filter (Sequential Monte-Carlo)

Sequential Monte-Carlo, Condensational Filter algorithm, Auxiliary Particle Filter [Youku]

(http://v.youku.com/v_show/id_XMTM3MTE1Mjk2OA)**

Advanced Probabilistic Model 高级概率模型课件

• Bayesian Non Parametrics (BNP) and its inference basics

Dircihlet Process (DP), Chinese Restaurant Process insights, Slice sampling for DP [dirichlet_process.m] and (http://v.youku.com/v_show/id_XMTM3NDY0MDkxNg) and [Jupyter Notebook]

[dirichlet_process.m] 和 [优酷链接] 和 [Jupyter Notebook]

• Bayesian Non Parametrics (BNP) extensions

Hierarchical DP, HDP-HMM, Indian Buffet Process (IBP)

非参贝叶斯扩展: 层次狄利克雷过程，分层狄利克雷过程-隐马尔可夫模型，印度自助餐过程(IBP)

• Determinantal Point Process

explain the details of DPP’s marginal distribution, L-ensemble, its sampling strategy, our work in time-varying DPP

行列式点过程解释:行列式点过程的边缘分布，L-ensemble，其抽样策略，我们在“时变行列式点过程”中的工作细节

Special Thanks

• I would like to thank my following PhD students for help me proofreading, and provide great discussions and suggestions to various topics in these notes, including (but not limited to) Hayden Chang, Shawn Jiang, Erica Huang, Deng Chen, Ember Liang; 特别感谢我的博士生团队协助我一起校对课件，以及就课件内容所提出的想法和建议，团队成员包括（但不限于）常浩东，姜帅，黄皖鸣，邓辰，梁轩。

• Special thanks to Dr Haiguang Huang for his efforts to translate my content into Chinese 特别感谢黄海广博士协助我将课件目录翻译成中文

• I always look for high quality PhD students in Machine Learning, both in terms of probabilistic model and Deep Learning models. Contact me on YiDa.Xu@uts.edu.au

如果你想加入我的机器学习博士生团队或有兴趣实习, 请通过YiDa.Xu@uts.edu.au与我联系。