SenteraLLC/geoml

Use skopt.gp_minimize for hyperparameter tuning optimization

tnigon opened this issue · 0 comments

Anything we've done thus far has been tuned using GridSearchCV, which is limited in two major ways:

  1. All hyperparameter results have to be stored then queried to find the optimal hyperparameters.
  2. GridSearchCV tuning does not inherently allow parallel processing, so it has to be implemented in a custom way (which is prone to bugs).

Instead, skopt.gp_minimize can be used to perform Bayesian optimization using Gaussian Processes. This will provide an optimal result for a given model (e.g., PLSR) and specific number of features from a feature set.

Implement skopt.gp_minimize in the hyperparameter tuning class.