GPOPY was build using a genetic algorithm, that will handle parameter optimizations for you in any problem
Gpopy now can be installed using pip
pip install gpopy
Or cloning the repository and importing the file from there on your project
#taken from model.py example
PARAMS = {
'batch_size' : [ 8,16,32,64,128,256],
'epochs' : [8, 16, 32, 64],
'dense_layers' : [64, 128, 256, 512],
'dropout' : {
'func' : random.uniform,
'params' : [0.3, 0.7]
},
'activation' : 'relu',
'learning_rate': {
'func' : random.uniform,
'params' : [0.01, 0.0001]
},
'filters' : [10, 64, 128],
'use_bias' : [True, False]
} params = {
'list_parameter' : [1,2,3,4,5] ,
'tupple' : (1,2)
}
#may return any value in the list and tuppleWhen running a list and/or tuple parameter, GPOPY will select from one of the values from then.
PARAMS = {
'random_function' : random.random,
}
#gpopy will evaluate random.random at each iteration
When passing a function without arguments, GPOPY will evaluate the function in each iteration, and use the result as parameter to pass to your function.
#random.uniform(0.01, 0.02)
PARAMS = {
'dict_func_object' : {
'func' : random.uniform,
'params' : [0.01, 0.02]
}
}The only dict we currently accept is with a function with a multiple parameters. This was created to make it easier to call functions that need parameters, as for example, random.uniform. The parser inside the code need the params to be 'func' and 'params.
If you use other type of values, for example int, double, string, they will be returned. This type is good when you already now a parameter that you don't want to change.Tunning(params, population_size = 4, maximum_generation = 20, mutation_rate = 0.25)params: Dict with all the params that will be used to generate your population
score: Function that will use as parameter the individue and return an score (e.g Keras model)
population_size: Number of unique individues generated at each generation (not included crossover and mutation individue)
maximum_generation: number of generations that will be run (in case you use run method)
mutation_rate : rate at which mutation occur, lower mutation_rate means more mutations
FlowTunning(pararms, polulation_size=2, maximum_generation=20, experiment_name="", auto_track=True) auto_track: If true, gpopy will track the values and evolution of your model. If FALSE, means you have an mlflow routine inside your model, and gpopy will only TAG it for you (mlflow.set_tag)
#Automaticly running
tunning = Tunning(PARAMS, population_size = 2)
tunning.set_score(model)
results = tunning.run() #will run model with a generation_number of 20tunning = FlowTunning(PARAMS, population_size=2)
tunning.set_score(model)
results = tunning.run() tunning = Tunning(PARAMS, population_size = 2)
tunning.set_score(model)
needed_accuracy = 0.95
for i in range(10):
"""Will run 10 times or until find the needed accuracy """
tunning.gen_population() #create new individue/mutation/crossovers
tunning.score() #run score functions and save the results from them
if tunning.top_score >= needed_accuracy:
break
top_score = tunning.top_score
best_match = tunning.first_parent
second_best_match = tunning.second_parentGPOPY has in-line documentation, few free to read and change it when needed, all tell us about your changes so we can keep getting better!
In order to be a score functions, we just need it to receive our data as parameter and return a score.
In this implementation of the algorithm, we will get the highest score for evolving.
Later we can change it
PARAMS = {
'a' : [1,2,4,8,16,32] ,
'b' : {
'func' : random.rantint,
'params' : (0, 100)
'x' : random.random
}
def model(data):
#unwrapping
a = data['a']
b = data['b']
x = data['x']
return a * (x ** 2) + b * x
}Since most of ML and DL are instantiated with random weights, some times under the same circunstance (PARAMS) we can obtain different test results
For resolving this, we added a option to "score" function to save models and reuse it later
tunning.score(save_model= True)
#or just
#tunning.score(True)
score, model = tunning.top_model #HA
model.evaluate(x_test, y_test, verbose=0)IF save_model= True, your score function should be returning a tupple, that constains (score, model), been model any function or value that you want to hold on memory
#model function returning from Keras
return (score[1], model)GPOPY now support the use of MLFlow https://mlflow.org/, in two different routines.
The behaviour of the two routines are similar to the previous one, with exception of using mlflow to track the results
We will enter in more details bellow
In this option, we will track only the best individue of each generations, and log the parameters used for optimizing this generation
You can use this option when you don't want advanced, or, a lot of visualizations, so you keep track of only GPOPY parameters and the result from your score function
#Example
from gpopy.gpopy import FlowTunning
#Your code and params here
#You can see this example at examples/simple_track_model.py
tunning = FlowTunning(params=PARAMS, population_size=4)
tunning.set_score(model)
tunning.run()While running your code, you have to access mlflow user interface.
mlflow ui
Will open an server, running on localhost:5000. For more informations of how to change this you can check mlflow documentation: https://mlflow.org/docs/latest/index.html
When accessing localhost:5000 (or whenever you set)
Here you can compare your models and observe parameters and results
And observe the evolution, generation per generation
In this option, we will stil optimize for you, and handle your parameters, but you have to keep a tracker on your model codes.
##Taked from examples/mlflow_fullmodel.py
tunning = FlowTunning(params=PARAMS, population_size=2, maximum_generation=5, auto_track=False)
tunning.set_score(model)
tunning.run() In this case, while declaring our model, we activate the mlflow auto tracking option.
Mlflow currently accept the most common ML and DL libraries, for using this, you can check mlflow docs and the example below
This versions stil tracks the models via mlflow ui, but now we have more informations as shown bellow
And this time, gpopy will not handle your models in memory, since mlflow save it for you
Mlflow tracks all the models that was running while optimizing, you can check if via Generation/Individue tags
You can have more advanced statistics extracted from your model, and compare with other models
And keep track of generation evolution
This module was added to help users to visualize what is happening, and in case of need, keep improving their model when gpopy was not sufficient
As was told earlier, GPOPY run Genetic algorithm on ANY function that meet those especifications, not only for ML and Neural Net Models.
* Genetic Algortihm optimzation with models in memory * Two optimizations using mlflow * Initial Model option * More crossover and mutation methods * Hybrid methods for optimization * A more clean organization of the files * More examplesFew free to use, edit and make all the changes needed in the code
If you use it in your work, please give us a credit
If you have any changes and/or bug to report, please few free to talk with us
You can personally contact me at lu.guerra7508@gmail.com feel free to be in touch
GPOPY is part of one paper i'm writing, so much more optimization and changes are coming
THANK YOU