DeepEnsemble
Keras ensembles, made easy.
Usage
Build or load as many models as you'd like. Ensembles of different model architectures tend to perform better than ensembles of the same model architecture.
#inception
model1 = models.load_model(
'models/model1.h5')
#custon (VGG like)
model2 = models.load_model(
'models/model2.h5')
#InceptionResNet
model3 = models.load_model(
'models/model3.h5')
Next, put them together in an ensemble
ensemble = [model1, model2, model3]
d = DeepEnsemble(ensemble, pos_label=1)
print(f'This is a {d.model_type} model.\n')
Always use DeepEnsemble with your validation set.
val_datagen = ImageDataGenerator(rescale=1./255)
batch_size_ = 1871
val_gen = val_datagen.flow_from_directory('learning_ready_data/val', batch_size=batch_size_, classes=[
'equip', 'no_equip'], target_size=(256, 384), class_mode='binary')
samples, sample_classes = val_gen.next()
Evaluate your ensemble on the validation set, using voting or average and with an option to
find and use an optimal threshold.
eval_as = 'average'
print(f'Evaluating model with {eval_as}')
d.evaluate(samples, sample_classes, eval_type=eval_as, optimal=True)
Look at various aspects of model performance
print(d._confusion_matrix)
print(f'Model accuracy scores {d.acc_scores}')
d.plot_confusion(show=True)
d.plot_roc(show=True)
Use two methods to find an weighted average of models that performs best on the validation set. You can use differential evolution with .find_weighted_avg() or a meta-learner, a one layer neural network with the output of the sub-models (probability scores) as the input to the meta-learner. The meta-learner can be accessed with .train_meta_learner().
print('Finding weighted average')
d.find_weighted_avg()
print('Training meta-learner')
d.train_meta_learner()
Output:
Optimized Weights: [0.32454564 0.32321557 0.35223879]
Optimized Weights Score: 0.98931
{'model0': 0.96044895777659,
'model1': 0.9428113308391235,
'model2': 0.9583110636023516,
'ensemble': 0.9887760555852485,
'weighted_ensemble': 0.9893105291288081}
Finally, predict using the ensemble with
d.predict(samples)
Contributions
Contributions are welcome! Please create an issue or a PR to collaborate on this library.