ulstu/ml

Jupyter Notebook

Training materials for the course "Machine learning"

Russia, Ulyanovsk Ulyanovsk State Technical University

1. Basics of Machine Learning

Brief history of basic technologies
Definitions
Basics of ML

2. Python

Basic syntax
Arithmetic
Strings
Lists
Exploratory data analysis with Pandas
Data visualization
Kaggle competitions
Titanic task

3. Linear regression

Linear regression
Cost function
Gradient descent
Linear regression with multiple variables
Debug (learning rate)
Normal equation
Features and polynomial regression
Multi-class classification
Feature scaling

4. Decision trees. KNN. Logistic regression. Regularization.

Decision tree
K-nearest neighbors
Classification problem
Sigmoid function
Decision boundary
Cost function
Regularization. Problem of overfitting

5. Machine learning system design

Error analysis. Metrics
Evaluating hypothesis. Train / test / validation set
High bias / high variance (model selection, regularization, learning curves)
Feature extraction:

Images
GEO
Date and time
Timeseries
Texts. One-hot encoding

6. Clustering

Clustering (k-means, c-means, hierarchical clustering)
Principal component analysis

7. Naive bayes and SVM

Bayes theorem
Naive bayes
Support vector machines

8. Neural networks

Non-linear hypothesis
Neurons and the brain
Forward propagation (XNOR example)
Back propagation
Parameters initializing

9. Deep learning. Convolutional neural networks

Convolution. Feature representation as hierarchy
Filters, stride, padding
Pooling
Popular architectures: AlexNet, VGG, ResNet,
Classification, localization, regression

10. Recurrent neural networks

Basics of recurrent NN
LSTM
Time-series analysis
Text analysis

11. Time seriaes analysis

12. Trees

Decision trees
Random forest
XGBoost
CatBoost

Links

Visual Attention Model