mlr-org/mlrMBO

negative se values in opt.path might confuse users

Closed this issue · 1 comments

Hello! :)
I would like to add only a short remark on the optimization path in mbo.result: With infill.crit = se, mbo() stores negative standard errors in the optimization path (see example below). This is because mbo() maximizes negative values internally and returns them in opt.path. For other infill criteria (such as EI) this might not be a problem, but for the standard error it might confuse users as standard errors by definition are positive.
I was wondering why not setting opt.direction = "minimize" and use positive function values? in makeMBOInfillCritStandardError ? Thanks in advance :)

Best
Julian


library(mlrMBO)
library(mlr)
library(smoof)

set.seed(010420)
obj.fun = makeCosineMixtureFunction(1)
des = generateDesign(n = 5, par.set = getParamSet(obj.fun), fun = lhs::randomLHS)
des$y = apply(des, 1, obj.fun)
surr.km = makeLearner("regr.km", predict.type = "se", covtype = "matern3_2", control = list(trace = FALSE))
control = makeMBOControl()
control = setMBOControlTermination(control, iters = 5)
control = setMBOControlInfill(control, opt = "focussearch", crit = makeMBOInfillCritStandardError())

mbo(obj.fun, design = des, learner = surr.km, control = control, show.info = TRUE)



Hi,
the SE-Infill in theory is maximized, that is why the value says "maximize". You want to evaluate points where the uncertainty is high. Internally all infill functions are minimized and that are the values we put into the opt.path. The same goes for the EI and others. The "maximize" tag is just used for plotting (to flip the values). We could do this in the opt.path as well, but we put our efforts in mlr3mbo so I will label this as wontfix if nobody complains.