Make specificity available as a CV metric
Closed this issue · 1 comments
Sensitivity and Specificity are the default standards for evaluating diagnostic tests (https://en.wikipedia.org/wiki/Sensitivity_and_specificity), sensitivity is called "recall" but specificity is not provided in https://scikit-learn.org/stable/modules/model_evaluation.html#classification-metrics but it can be obtained through the confusion matrix or the recall of the negative class.
Implement in way that can be incorporated in the cv_metrics field of a classifier (i.e. "accuracy, recall, specificity"):
- https://github.com/math-a3k/django-ai/blob/covid-ht/django_ai/supervised_learning/models/supervised_learning_technique.py#L52
- https://github.com/math-a3k/django-ai/blob/covid-ht/django_ai/supervised_learning/models/supervised_learning_technique.py#L163
- https://github.com/math-a3k/django-ai/blob/covid-ht/django_ai/supervised_learning/models/supervised_learning_technique.py#L354
- https://github.com/math-a3k/django-ai/blob/covid-ht/django_ai/supervised_learning/metrics.py
get_cross_validation_scores() is currently considered to be engine specific:
More info: https://scikit-learn.org/stable/modules/model_evaluation.html#scoring-parameter
This hit a limitation / flaw in django-ai (math-a3k/django-ai@fd85385), with that settled, it is implemented in fbfb983.