jerpint/hydra_orion_sweeper

Hydra Sweeper Plugin for Orion

Hydra Sweeper plugin for Orion

Provides a mechanism for Hydra applications to use Orion algorithms for the optimization of the parameters of any experiment.

See website for more information

Install

pip install hydra-orion-sweeper

Search Space

Orion defines 5 different dimensions that can be used to define your search space.

• uniform(low, high, [discrete=False, precision=4, shape=None, default_value=None])
• loguniform(low, high, [discrete=False, precision=4, shape=None, default_value=None])
• normal(loc, scale, [discrete=False, precision=4, shape=None, default_value=None])
• choices(*options)
• fidelity(low, high, base=2)

Fidelity is a special dimension that is used to represent the training time, you can think of it as the epoch dimension.

Documentation

For in-depth documentation about the plugin and its configuration options you should refer to Orion as the plugin configurations are simply passed through.

• algorithm
• worker
• storage
• parametrization

Example

defaults:
- override hydra/sweeper: orion

hydra:
sweeper:
   params:
      a: "uniform(0, 1)"
      b: "uniform(0, 1)"

   orion:
      name: 'experiment'
      version: '1'

   algorithm:
      type: random
      config:
      seed: 1

   worker:
      n_workers: -1
      max_broken: 3
      max_trials: 100

   storage:
      type: legacy
      database:
         type: pickleddb
         host: 'database.pkl'

# Default values
a: 0
b: 0

import hydra
from omegaconf import DictConfig

@hydra.main(config_path=".", config_name="config")
def main(cfg: DictConfig) -> float:
   """Simple main function"""
   a = cfg.a
   b = cfg.b

   return float(a + b)

if __name__ == "__main__":
   main()