Welcome to the open source repo of The Little Book of ML Metrics. The idea of the book is to be this little handbook that sits on every data scientist's desk for quick reference, from the most well-known metric, ahem, accuracy, to the most obscure ones (looking at you, P4 metric).
Machine learning metrics are often overlooked in traditional data science courses and university degrees. This book aims to fill that gap by offering clear, concise, and actionable explanations of the metrics that matter most. Whether you're an aspiring data scientist or an experienced professional, this book will become your go-to reference for understanding and leveraging metrics effectively.
Disclaimer: The book is open-source, which means you can freely access the digital version. However, we also offer a high-quality printed edition for purchase. Revenue from printed copies helps support further development and maintenance of the book. Reviewers, contributors, and authors receive revenue sharing through their affiliate links.
The book covers a broad range of metrics from different contexts:
- Regression
- Classification
- Clustering
- Ranking
- Computer Vision
- NLP
- GenAI
- Probabilistic
- Bias and Fairness
- Business
We welcome contributions from the community! As a thank-you for your contributions, each contributor will receive an affiliate link with 10% commission on sales generated through their link. Plus, your name will be included in the book. Please check our Contributing Guidelines for more details.
If you're an expert in any of the topics described in the book contents section and would like to review this book, please fill out this form. As a thank-you, reviewers will receive an affiliate link with 15% commission on sales generated through their link. Plus, your name will be included in the book.
Santiago Viquez
ML Developer Advocate at NannyML. Santiago has over five years of professional experience in ML and data science. He holds a Bachelor’s degree in Physics and a Master’s degree in Data Science.
Wojtek Kuberski
Co-founder and CTO at NannyML. Wojtek is an AI professional and entrepreneur with a Master’s degree in AI from KU Leuven. He co-founded NannyML, an OSS Python library for ML monitoring and post-deployment data science. As the CTO, he leads the research and product teams, contributing to the development of novel algorithms in model monitoring.
This project is backed by NannyML, the only platform for monitoring machine learning models in production that can estimate model performance metrics without ground truth.