purepisces/Wenqing-Machine_Learning_Blog

Aiming to build the most comprehensive machine learning blog.

Current progress is updated in progress.txt

中文版欢迎关注我的CSDN

Wenqing's Machine Learning Blog

Foundational Concepts in Machine Learning

• Linear Algebra

  • Basic
  • Numpy Tips
  • Inverse matrix
  • Diagonal matrix
  • Determinants
  • Linear Transformations
  • Linear Systems
  • Change Of Basis
  • Eigenvector
  • Singular Vector
  • Eigenvalue
  • Singular Value
  • Singular Value Decomposition (SVD)

• Calculus

  • Calculus

• Statistical Measures

  • Standard Deviation vs Standard Error
  • Margin of Error
  • Statistical Inference
  • Confidence Intervals
  • Expected Value
  • Statistical Power
  • P-values
  • Statistical Significance
  • Bias
  • Variance&Standard Deviation
  • Covariance
  • Correlation
  • Joint Variability
  • Bessel’s Correction
  • Degrees of Freedom
  • Cardinality
  • Entropy
  • Probability Distributions
    • Uniform Distribution
    • Normal Distribution
    • Cumulative Distribution Function (CDF)
    • Standard Gaussian Distribution
    • Bernoulli Distribution
    • Binomial Distribution

• Feature Engineering

  • Overview of Feature Engineering
  • One Hot Encoding
  • Feature Hashing

• Feature Selection

  • Overview of Feature Selection

• Model Evaluation

  • Model_Eval
  • Offline Metrics
    • Underfitting (High Bias) & Overfitting (High Variance)
    • AUC (Area Under Curve)
    • logloss
    • RMSE
    • MAPE
    • Confusion Matrix
    • Precision, Recall and F1 score
  • Online Metrics

• Training Pipeline

  • Training Pipeline Overview
  • Data Partitioning
  • Handle Imbalance Class Distribution
  • Choose the Right Loss Function
  • Retraining Requirements

• Linear Regression

  • Overview
  • Assumptions of Linear Regression
    • Linearity
    • Homoscedasticity (Constant Variance)
    • Normal Distribution of Errors
    • No Multicollinearity
    • Independence of Errors(Non-Random Residuals)
  • Performance Metrics for Linear Regression
    • R-squared
    • P-values
    • Statistical Significance
    • Prediction Interval
    • Standard Error of the Estimate

• Logistic Regression

  • Overview
  • Softmax-Regression

• Decision Trees

  • Overview

• T-SNE

  • Overview

• Dimensionality Reduction

  • Principal Component Analysis (PCA)

Deep Learning Concepts

Basic Neural Network Concepts

• Neural Network

  • Overview

• Feed Forward Neural Network (FFNN)

  • Overview
  • MLP (Multi-Layer Perceptron)
    • Overview

• Activation Functions

  • Overview
  • ReLU
  • Sigmoid
  • Tanh
  • GELU
  • Softmax

• Backpropagation

  • Overview

• Neural Network Layers

  • Linear Layer
  • Residual Connection
  • ReLU Activation
  • Flatten Layer
  • Batch Normalization (BatchNorm1d)
  • Layer Normalization (LayerNorm1d)
  • Dropout
  • Identity Layer
  • Sequential Layer

• Neural Network Models

  • Overview
  • MLP
  • ResNet

• Loss Functions

  • Overview
  • Classification Tasks:
    • Cross Entropy Loss
      • Binary Cross Entropy Loss
      • Normalized Cross Entropy
    • Ranking Loss
  • Regression Tasks:
    • MSE Loss
    • MAPE
    • SMAPE
    • Quantile Loss

• Optimization Algorithms

  • Overview
  • Gradient Descent
  • Batch Gradient Descent
  • Stochastic Gradient Descent (SGD)
  • Stochastic Gradient Descent with momentum (SGD with momentum)
  • Mini-Batch Gradient Descent
  • Adam
  • Advanced Optimization Algorithms
    • Momentum
    • Adam
    • RMSProp
    • AdaGrad

• Regularization

  • Overview
  • L1 Regularization
  • L2 Regularization
  • Dropout
  • Early Stopping

• Normalization

  • Batch Normalization
  • Layer Normalization

Hardware and Parallel Processing

• CPU vs GPU Performance
• CUDA Kernels and GPU Parallelism

Advanced Neural Network Architectures

• Convolutional Neural Networks (CNN)

  • Overview of CNNs
  • Convolutional Layers
  • Pooling Layers
  • Stride and Padding
  • CNN Architectures (e.g., LeNet, AlexNet)

• Recurrent Neural Networks (RNN)

  • Overview of RNNs
  • LSTM (Long Short-Term Memory)

• Transformer Networks

  • Overview of Transformers

Sequence Models

• Overview
• Language Model Evaluation Metrics
  • Perplexity
  • [Other relevant metrics...]
• RNN (Recurrent Neural Networks)
  • Standard RNN
  • LSTM (Long Short-Term Memory)
• CNN (Convolutional Neural Networks)
  • Overview
• Transformers
  • Original Transformer
  • BERT (Bidirectional Encoder Representations from Transformers)
    • Overview
  • GPT (Generative Pre-trained Transformer)
    • Overview
  • T5 (Text-to-Text Transfer Transformer)
    • Overview
• Model Comparisons: CNN vs. RNN vs. Transformer

Large Language Models

• Overview

Generative AI

• Overview
  • Overview
• Generative Adversarial Networks (GANs)
  • Introduction to GANs
• Variational Autoencoders (VAEs)
  • Introduction to VAEs
• Applications of Generative AI
  • Applications

Word Representations

• Embedding
  • Overview

Machine Learning System Design Notes From Educative

• Video Recommendation
  • Matrix Factorization
  • Collaborative Filtering
• Feed Ranking
• Ad Click Prediction
• Rental Search Ranking
• Estimate Delivery Time

Code Implementation

• Image Segmentation Task For CGG
• Decision Tree
• Logistic Regresssion
• Softmax Regresssion

Machine Learning Interview Questions

• Interview Questions