Model Selection & Hyper-parameter selection

[eric-haibin-lin]

CNNs share the same fate with other Neural Nets. There are too many parameters to tune; optimal input patch size, number of hidden layers, number of feature maps per layer, pooling and stride sizes, normalization windows, learning rate and others. Thus, the problem of model selection is relatively harder compared to other ML techniques.

LibSVM comes with a nice script grid.py that performs grid search to optimize hyper-parameter for model selection. Is it possible to equip this toolbox with such tool as well? Or, it's better to utilize another model selection library to optimize hyper-parameter?

Thanks.

[tambetm]

Some people have had good experience with https://www.whetlab.com/. Even if it doesn't optimize all parameters automatically, at least it gives you insight which hyperparameters actually matter and which don't. There is nice API for Matlab.