AnanduR32/MachineLearning-Basics

ML dataScience Basics

• Courses to learn Datascience :
  • https://app.dataquest.io
  • https://lambdaschool.com/courses/data-science

Prerequisites :

• https://www.khanacademy.org/math/linear-algebra
• https://www.khanacademy.org/math/statistics-probability
• https://www.khanacademy.org/math/ap-statistics

Important Architectures/Algorithms

Basics

• Feature Scaling : (The contour analysis shows an extremely skewed figure in case the parameters are on varying scales, which causes convergence to be extremely slow and inefficient)
  • https://medium.com/greyatom/why-how-and-when-to-scale-your-features-4b30ab09db5e
• Regularization :
  • https://towardsdatascience.com/regularization-in-machine-learning-76441ddcf99a
• Intuition behind gradient descent :
  • https://towardsdatascience.com/machine-learning-bit-by-bit-multivariate-gradient-descent-e198fdd0df85
• Bayes Theorm :
  • https://en.wikipedia.org/wiki/Bayes%27_theorem
• Decision Matrix :
  • http://web.iitd.ac.in/~sumeet/mlintro_doc.pdf

Logistic regression

• Line Search algorithm - *To create efficient classification models like "Conjugate gradient", "BFGS", and "L-BFGS" ...
  • https://www.ii.uib.no/~lennart/drgrad/More1994.pdf

Neural Networks

• Derivation of backpropagation Algorithm (Hardest math in ML):
  • https://medium.com/binaryandmore/beginners-guide-to-deriving-and-implementing-backpropagation-e3c1a5a1e536
  • https://sudeepraja.github.io/Neural/
• Optimal brain damage algorithm (CNN compression) :
  • http://yann.lecun.com/exdb/publis/pdf/lecun-90b.pdf
• Cascade correlation (network) learning architecture :
  • http://papers.nips.cc/paper/207-the-cascade-correlation-learning-architecture.pdf
• Avoiding local optima using simulated annealing :
  • https://en.wikipedia.org/wiki/Simulated_annealing
  • http://www.cs.cmu.edu/afs/cs.cmu.edu/project/learn-43/lib/photoz/.g/web/glossary/anneal.html
• Parameter Estimation (Maximum Likelihood Estimation):
  • http://statweb.stanford.edu/~susan/courses/s200/lectures/lect11.pdf

Decision Trees

• Regression Trees - Model Trees :
  • https://towardsdatascience.com/ensemble-methods-bagging-boosting-and-stacking-c9214a10a205
  • https://towardsdatascience.com/introduction-to-model-trees-6e396259379a
• Cross Validation (Very Important) :
  • https://towardsdatascience.com/cross-validation-in-machine-learning-72924a69872f
• Bagging and boosting :
  • https://towardsdatascience.com/decision-tree-ensembles-bagging-and-boosting-266a8ba60fd9 Boosting is a very powerful tool works really well with decision trees
  • Random Forests -
    • https://towardsdatascience.com/understanding-random-forest-58381e0602d2

Evaluation/ Evaluation measures

• Bootstrapping :
  • https://towardsdatascience.com/an-introduction-to-the-bootstrap-method-58bcb51b4d60
• Explaination to Accuracy, Recall, Precision, F-Score, Specificity and sensitivity :
  • https://towardsdatascience.com/accuracy-recall-precision-f-score-specificity-which-to-optimize-on-867d3f11124
• Supervised-Learning : Ranking :
  • https://medium.com/@nikhilbd/intuitive-explanation-of-learning-to-rank-and-ranknet-lambdarank-and-lambdamart-fe1e17fac418
  • https://towardsdatascience.com/learning-to-rank-with-python-scikit-learn-327a5cfd81f

Graphical Models

• Basics
  • https://lili-mou.github.io/resource/MarkovNet.pdf important - contains condensed content on everything
  • http://mlg.eng.cam.ac.uk/zoubin/talks/lect2gm.pdf
• Markov Blanket for directed and undirected graphs
  • https://library.bayesia.com/display/FAQ/Markov+Blankets
    • Markov networks can be used to factorize dependencies through which we can remove unnecessary dependencies from an equation
• Hammersley–Clifford theorem
  • https://en.wikipedia.org/wiki/Hammersley%E2%80%93Clifford_theorem
• Bayesian Networks :
  • http://www.cs.ru.nl/~peterl/teaching/CI/bbn1-new4.pdf
• Representation of Undirected Graphical Model
  • https://www.cs.cmu.edu/~epxing/Class/10708-17/notes-17/10708-scribe-lecture3.pdf
• Hidden Markov Model - useful in NLP context
  • https://medium.com/@kangeugine/hidden-markov-model-7681c22f5b9
  • https://medium.com/greyatom/learning-pos-tagging-chunking-in-nlp-85f7f811a8cb
  • Chunking :
    • https://nlpforhackers.io/text-chunking/

Clustering

• https://towardsdatascience.com/clustering-on-mixed-type-data-8bbd0a2569c3

Partitional

• K-Means :
  • http://scikit-learn.org/stable/modules/generated/sklearn.cluster.KMeans.html
  • https://towardsdatascience.com/understanding-k-means-clustering-in-machine-learning-6a6e67336aa1
• K-Medoids :
  • https://www.geeksforgeeks.org/ml-k-medoids-clustering-with-example/

Hierarchical

Density-Based

Gaussian Mixture Models

• https://scikit-learn.org/stable/modules/mixture.html
• https://towardsdatascience.com/gaussian-mixture-models-explained-6986aaf5a95
• http://www.cse.iitm.ac.in/~vplab/courses/DVP/PDF/gmm.pdf

Reinforcement learning

• Temporal difference learning
  • https://medium.com/@violante.andre/simple-reinforcement-learning-temporal-difference-learning-e883ea0d65b0

Learning Theory

• Error Generalization
  • https://medium.com/@yixinsun_56102/understanding-generalization-error-in-machine-learning-e6c03b203036
• Emperical Error Minimization :
  • https://towardsdatascience.com/learning-theory-empirical-risk-minimization-d3573f90ff77
  • (Optional) https://towardsdatascience.com/what-is-empirical-risk-minimization-erm-ef9edc76b48
• Vapnik - Chenrvenenkis Dimensions :
  • https://towardsdatascience.com/measuring-the-power-of-a-classifier-c765a7446c1c

Image classification Models

• Image classification using Tensorflow
  • https://medium.com/@tifa2up/image-classification-using-deep-neural-networks-a-beginner-friendly-approach-using-tensorflow-94b0a090ccd4
• Image classification using Keras
  • https://www.pyimagesearch.com/2017/12/11/image-classification-with-keras-and-deep-learning/
• Machine Learning practice
  • https://www.hackerearth.com/practice/machine-learning/machine-learning-algorithms/beginners-guide-regression-analysis-plot-interpretations/tutorial/