Inclusion of more metrics

[StijnKas]

I have always used metrics such as Precision, Recall and F-Score in conjunction with AUC score. I think these metrics are important to consider if there is class imbalance. Precision, recall, and F-score provide more specific information regarding how well the model performs in correctly identifying a specific class. AUC, while helpful in measuring the model's ability to distinguish between classes, does not provide class-specific details.

Since they are already included in R and my reasoning above I hope this provides a valid use case to include them in the python scripts as well. :)

Ideally these are the type of stats I would like to see to compare model performance when we perform experimentation:

Originally posted by @sushantkhare in #153 (reply in thread)

[operdeck]

Just remember the ADM models do not predict 0/1 but a probability. Op di 7 nov. 2023 16:08 schreef Stijn Kas ***@***.***>:

…

I have always used metrics such as Precision, Recall and F-Score in conjunction with AUC score. I think these metrics are important to consider if there is class imbalance. Precision, recall, and F-score provide more specific information regarding how well the model performs in correctly identifying a specific class. AUC, while helpful in measuring the model's ability to distinguish between classes, does not provide class-specific details. Since they are already included in R and my reasoning above I hope this provides a valid use case to include them in the python scripts as well. :) Ideally these are the type of stats I would like to see to compare model performance when we perform experimentation: [image: f217464c-6b13-4bbf-9a12-28173bcf1f6c] <https://user-images.githubusercontent.com/69606569/279380123-308a4f6c-7c6a-4826-b789-bb843da2c2aa.jpg> *Originally posted by @sushantkhare <https://github.com/sushantkhare> in #153 (reply in thread) <#153 (reply in thread)>* — Reply to this email directly, view it on GitHub <#165>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABEVTKET7FHLUCFV2U3KZ5LYDJFF5AVCNFSM6AAAAAA7BLCUBCVHI2DSMVQWIX3LMV43ASLTON2WKOZRHE4DCNJYGYZTIMQ> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[StijnKas]

I don't think there is enough demand for this and it's very easy for these kind of evaluation metrics to be misleading in our setup where we have probability models in a ranking scenario. Closing for now, if anyone feels a need for this I'm open to reconsider.