sherpa-ai/sherpa

Population Based Training algorithm: Difference between parameters and parameter_range

martsalz opened this issue · 1 comments

Could you please explain the exact difference between parameters and parameter_range?
Why are the parameters specified twice?

image

https://parameter-sherpa.readthedocs.io/en/latest/algorithms/pbt.html

Thanks.

The parameter_range argument defines the range in which the parameter can evolve whereas for PBT the actual parameter definitions define on what range the parameters will be initialized. Here "evolve" corresponds to the fact that PBT perturbs parameters. So the user may want to set the range that values can evolve in to be larger than the initialization range. For example for learning rates we may only want to initialize within a certain range but then allow to be perturbed to much smaller values than that.