microsoft/MLOS

LlamaTune inverse_trainsform() produces invalid configuration

Closed this issue · 1 comments

LlamaTuneAdapter.inverse_transform() apparently produces small float values like 1.5 and 0.998 and assigns them to int hyperparameters with range [1, 10000].
If we add validation immediately after producing such config, it will fail, causing the corresponding unit test failure:

FAILED mlos_bench/mlos_bench/tests/optimizers/llamatune_opt_test.py::test_llamatune_optimizer - ConfigSpace.exceptions.IllegalValueError: Value 1.5: (<class 'float'>) is not allowed for hyperparameter dim_0, Type: UniformInteger, Range: [1, 10000], Default: 5000

Interestingly, in ConfigSpace 1.0 same float inputs get scaled up from [0, 1] to int range of the corresponding hyperparameter. That also produces invalid configuration because value of 1.5 got scaled to 15000, i.e., outside of the hyperparameter's [1, 10000] range.

@kkanellis can you please take a look at it? I am bit lost in LlamaTune code. Use #806 to reproduce the problem, then run

pytest -vv -n1 mlos_bench/mlos_bench/tests/optimizers/llamatune_opt_test.py::test_llamatune_optimizer

Thank you!