Init Function always defaults to RandomSampling?

[123epsilon]

Hi, I am experimenting with the CBoptimizer and wanted to use the GridSampling initialization for my model but I've found that limbo seems to always execute init_function_t() as RandomSampling. I tried using the provided init::GridSampling and a custom struct that replicates that functionality but the experiment always runs the RandomSampling initialization. When I observe the model in gdb it is clear that it is initialized with the initfun parameter set to init::GridSampling.

Here is how I am setting the initialization:

using Kernel_t = kernel::SquaredExpARD<Params>;
using Mean_t = mean::Data<Params>;
using HyperOpt_t = model::gp::KernelLFOpt<Params>;
using GP_t = model::GP<Params, Kernel_t, Mean_t, HyperOpt_t>;

//init conditional EI acquisition function
using kf_t = limbo::kernel::Exp<Params>;
using meanc_t = limbo::mean::Constant<Params>;
using constraint_model_t = limbo::model::GP<Params, kf_t, meanc_t, HyperOpt_t>;
using Acqui_t = experimental::acqui::ECI<Params, GP_t, constraint_model_t>;
using acquiopt_t = opt::NLOptNoGrad<Params, nlopt::GN_ISRES>;

using stat_t = GPViz<Params>;
using init_t = init::GridSampling<Params>; 
using stop_t = stop::CondGoalStop<Params>;


experimental::bayes_opt::CBOptimizer<Params, modelfun<GP_t>, acquiopt<acquiopt_t>, acquifun<Acqui_t>, experimental::constraint_modelfun<constraint_model_t>, stopcrit<stop_t>, initfun<init_t>, statsfun<stat_t>> boptimizer;
// run the evaluation
boptimizer.optimize(EvalRealSZ3Params_CompressRatio());

When I use gdb to look at the boptimizer object I get the following:

limbo::experimental::bayes_opt::CBOptimizer<Params, limbo::modelfun<limbo::model::GP<Params, limbo::kernel::SquaredExpARD<Params>, limbo::mean::Data<Params>, limbo::model::gp::KernelLFOpt<Params, limbo::opt::Rprop<Params> > > >, limbo::acquiopt<limbo::opt::NLOptNoGrad<Params, (nlopt::algorithm)35> >, limbo::acquifun<limbo::experimental::acqui::ECI<Params, limbo::model::GP<Params, limbo::kernel::SquaredExpARD<Params>, limbo::mean::Data<Params>, limbo::model::gp::KernelLFOpt<Params, limbo::opt::Rprop<Params> > >, limbo::model::GP<Params, limbo::kernel::Exp<Params>, limbo::mean::Constant<Params>, limbo::model::gp::KernelLFOpt<Params, limbo::opt::Rprop<Params> > > > >, limbo::experimental::constraint_modelfun<limbo::model::GP<Params, limbo::kernel::Exp<Params>, limbo::mean::Constant<Params>, limbo::model::gp::KernelLFOpt<Params, limbo::opt::Rprop<Params> > > >, limbo::stopcrit<limbo::stop::CondGoalStop<Params> >, limbo::initfun<limbo::init::GridSampling<Params> >, limbo::statsfun<GPViz<Params> > >

But when stepping through CBoptimizer::optimize using gdb I see that init_function_t(seval, afun, *this) always routes to the RandomSampling initialization, strangely I can not query this value in gdb as the symbol does not exist, though I have removed all optimization from the compilation process.

Is there something wrong in how I am trying to use this initialization scheme? Would appreciate any help with this.

[123epsilon]

Hi, just wondering if I could get some insight on this - its very strange that setting a different initialization scheme still calls the random sampling function (this is visible even in the list of samples we take).

[costashatz]

Hello @123epsilon!

Sorry for the very late reply.

Can you try this initialization instead?

experimental::bayes_opt::CBOptimizer<Params, modelfun<GP_t>, stopcrit<stop_t>, initfun<init_t>, statsfun<stat_t>, acquifun<Acqui_t>, experimental::constraint_modelfun<constraint_model_t>, acquiopt<acquiopt_t>> boptimizer;

Let me know if it calls the correct initialization function...

[123epsilon]

@costashatz Wow, yes this works - is the order here significant? I notice that this still not the same as the cboptimizer signature but permuting the order made this work - I'm unfamiliar with how boost templating works.

using cboptimizer_signature = boost::parameter::parameters<boost::parameter::optional<limbo::tag::acquiopt>,
                boost::parameter::optional<limbo::tag::statsfun>,
                boost::parameter::optional<limbo::tag::initfun>,
                boost::parameter::optional<limbo::tag::acquifun>,
                boost::parameter::optional<limbo::tag::stopcrit>,
                boost::parameter::optional<limbo::tag::modelfun>,
                boost::parameter::optional<limbo::experimental::tag::constraint_modelfun>>;

[costashatz]

@123epsilon this is because this code is experimental and we did not spend time to fully integrate this into the library. The order should never matter in the "fully integrated" classes/functions. Long story short, the code in BayesOptimizer is looking only in the first 6 template params for initialization structure. Thus if you put it in the end, it doesn't even see it :) In any case, if you are using the CBOptimizer and you think it's mature enough to integrate inside the actual library, I could find some time to finalize it.

Closing the issue.

[123epsilon]

@costashatz Thank you for the explanation that makes sense, I can't speak to the maturity of the implementation - but I have found limbo to be quite useful for the research I'm conducting using Bayesian Optimization and since we are dealing with a constrained optimization problem it would always be helpful to have a canonical constrained bayesian optimization routine in the library :)

I've spent quite a bit of time looking at the source for the CBOptimizer so if there are any implementation checks/code contributions that you need help with I'd be happy to contribute