Obtain additional output parameters from the objective function

[anamabo]

Hi,
I was exploring this library and it is very nice and intuitive to use! I have a question though:

In my objective function I'm calling another function that returns many variables, including the one I want to optimise. Is there a way in which I can save those results?

The code I'm using is something like the following snippet:

defined_space= {}

results = []
def funct(a,b,c):
do stuff
return loss, other_params

def objective(**params):
loss, other_params = funct(**params)
results.append(other_params)
return loss

tuner = Tuner(space, objective)

When I run mango, the list "results" is still empty even when I'm appending information every time I call the objective function. How can I save "other_params"?

Thank you a lot for your help!

[sandeep-iitr]

Thanks for asking the question. If I understand correctly, you need to store some of the intermediate information from the objective function. The following code works for me. You can try and let me know from your end.

from mango import scheduler, Tuner
param_space = dict(x=range(-10,10))

"""
Storing any form of results from the function
"""
results = []

@scheduler.serial
def objective(x):
global results

results.append(x) #adding intermediate information
return x * x

tuner = Tuner(param_space, objective)
tuner_results = tuner.minimize()

print(tuner_results["best_objective"])
print('Stored Results:',results)

[anamabo]

Hi @sandeep-iitr:

Thank you a lot for your prompt response. The example above indeed works; but when the scheduler function is parallel, the results list is empty. Is there an easy way to get values with the parallel configuration? I provide above an example:

param_space = dict(x=range(-10,10), y= range(-5,5))
results = []

def _funct(x, y):
    asum = x+y
    amult = x*y
    return asum, amult

def _objective(x, y):
    global results
    loss, other_param = _funct(x, y)
    results.append(other_param)
    return loss

@scheduler.parallel(n_jobs=2)
def objective2(**args):
    return _objective(**args)

tuner = Tuner(param_space, objective2)
tuner_results = tuner.minimize()
print(results)

As always, thank you for your response!

[sandeep-iitr]

Thanks. We will get back to you on this soon.

[tihom]

Hi @anamabo,

cc: @sandeep-iitr

Since the parallel scheduler uses joblib which spawn multiple processes to parallelize the execution. These parallel processes do not share the memory so the results.append(other_param) command does not modify the results object in the main process. For this use case you can write your own parallelized objective function like so:

from mango import Tuner
from joblib import Parallel, delayed

other_results = []

def _func(x, y):
    asum = x+y
    amult = x*y
    return asum, amult

n_jobs = 2

def objective(params_batch):
    global other_results

    results_batch = Parallel(n_jobs)(delayed(_func)(**params) for params in params_batch)

    other_results += [result[1] for result in results_batch]
    losses = [result[0] for result in results_batch]

    return losses

param_space = dict(x=range(10), y=range(10))

tuner = Tuner(param_space, objective, dict(num_iteration=4))
tuner_results = tuner.minimize()

print('Params: ', tuner_results['params_tried'])
#=> Params:  [{'y': 1, 'x': 8} {'x': 0, 'y': 9} {'x': 9, 'y': 9} {'x': 0, 'y': 0} {'x': 0, 'y': 1}]
print('Other results: ', other_results)
#=> Other results:  [8, 0, 81, 0, 0]
print('losses: ', tuner_results['objective_values'])
#=> losses:  [ 9  9 18  0  1]

Let us know if this works for you. Thanks.

[anamabo]

@tihom and @sandeep-iitr, thank you so much! This example is just what I was looking for. I implemented it and it works.