Dependent parameters

[djalusic]

Is it possible to use a param_space dictionary where one parameter depends on another? E.g. hidden layer dimensions on a number of layers in neural networks.

[sandeep-iitr]

Dear Dorotea Rajšel,

Thanks for asking this question.

I have created a simple example of a neural network using Keras on Iris data, the hidden layer dimensions depend on the number of hidden layers in the example.
The approach is to specify the independent dimensions in the param_space, and then the dependent dimensions can be handled in the objective function during evaluation. This approach will work for simple dependencies. Like for one-to-one dependencies. The example is available here: examples/NeuralNetwork_Simple.ipynb

In case, where dependent dimensions too have a complex search space specified on the independent dimensions, the above approach will not work. In this case, a hierarchical mango optimizer can be used, where within objective function, a new param_space can be defined based on the independent dimensions, to search through the dependent ones.

regards,
Sandeep Singh.

[djalusic]

Great, thank you for your detailed answer.