pouyaardehkhani/ParameterOptimization

In this project, we focus on different ways to optimize a machine learning model parameters.

ParameterOptimization

What I focus in this project:

1. Difference between parameters and hyper-parameters.
2. Different Metrics.
3. Different ways of Cross Validation.
4. Basic Search
   • Manual Search
   • Grid Search
   • Random Search
5. Bayesian Optimisation
   • Sequential Search
   • Trade-off
   • Statistics
   • Bayes’ Rule
   • Probability reallocation
   • Gaussian Distribution
   • Gaussian Process
   • Kernels
   • Acquisition Functions
   • Implementation
6. Other SMBO Algorithms
   • Bayesian Optimization with Random Forests (SMAC)
   • Tree-structured Parzen Estimators (TPE)
   • Search strategies
   • Annealing with Hyperopt
7. Scikit-Optimize
   • Search algorithms
   • objective function
   • Sklearn
   • Search Space
   • Acquisition Function
   • Analysis
   • parallelization
   • Implementation
8. Hyperopt
   • Search algorithms
   • objective function
   • Search Space Configuration
   • nested spaces
   • Acquisition Function
   • Analysis: Trials
   • Parallelization -MongoDB
   • Implementation
9. Optuna
   • Search algorithms
   • objective function
   • Distributions
   • Acquisition Function
   • Search analysis
   • Parallelization -SQLite
   • Main setup
   • Implementation