How to change `measure` and `resampling` and maximize kappa value?

[GegznaV]

In a HyperControl object, is there a way to change either measure or resampling (or both) and leave other settings at their defaults?

This code results in error:

makeHyperControl(resampling = cv3,  measures   = kappa)
## Error in makeHyperControl(resampling = cv3, measures = kappa) : 
## Assertion on 'mlr.control' failed: Must have class 'TuneControl', but has class 'NULL'.

And manual modification of fields result in unexpected outcome:

set.seed(1)
# Create the object:
h_ctrl <- generateHyperControl(task = sonar.task, learner = "classif.ranger")
# Manually modify the fields:
h_ctrl <- makeHyperControl(h_ctrl$mlr.control,
                           resampling = cv3, 
                           measures   = kappa,
                           par.config = h_ctrl$par.config)
# Tune
h_params <- hyperopt(sonar.task,
                     learner = "classif.ranger",
                     hyper.control = h_ctrl,
                     show.info = FALSE)

h_params
## Op. pars: mtry=1; min.node.size=2
## kappa.test.mean=-0.64

It seems that it was searched for the minimum kappa value

[GegznaV]

One more example with iris dataset:

h_ctrl <- generateHyperControl(task = iris.task, learner = "classif.ranger")

set.seed(1)
h_ctrl <- makeHyperControl(h_ctrl$mlr.control,
                           resampling = cv3, 
                           measures   = kappa,
                           par.config = h_ctrl$par.config)
h_params <- hyperopt(iris.task,
                     learner = "classif.ranger",
                     hyper.control = h_ctrl,
                     show.info = FALSE)
set.seed(1)
h_ctrl2 <- makeHyperControl(h_ctrl$mlr.control,
                           resampling = cv3, 
                           measures   = mmce,
                           par.config = h_ctrl$par.config)

h_params2 <- hyperopt(iris.task,
                     learner = "classif.ranger",
                     hyper.control = h_ctrl2,
                     show.info = FALSE)
h_params
## Tune result:
## Op. pars: mtry=4; min.node.size=5
## kappa.test.mean=-0.938

h_params2
## Tune result:
## Op. pars: mtry=1; min.node.size=7
## mmce.test.mean=0.0533

"Good" mmce value, but "bad" kappa for the same task seems suspicious.

[jakob-r]

Thanks for spotting that bug. I have to fix it in mlr. But it is just an output bug. Measures that are to be maximized are transformed with -1 because the optimizer always minimizes.