kumar-abhishek/awesome-imbalanced-learning-in-production

Awesome Imbalanced Learning in Production

If you find this content insightful, feel free to support with a ⭐

A search for "Imbalanced data" on Google Scholar yields over one million results in total, with approximately 17,000 results since 2023. This raises a question: Is it merely a popular topic for academic papers, or is it a genuine challenge faced in the industry?

This GitHub repository offers a curated list of resources demonstrating how various companies have successfully tackled the challenge of imbalanced data in production settings. This repository is aimed at being an invaluable resource for professionals/researchers/enthusiasts in machine learning and data science dealing with similar challenges.

Inspired By Awesome-System-Design

Contents

• Oversampling Methods
• Undersampling Methods
• Ensemble Methods
• Cost-Sensitive Learning
• Data Imbalance in Deep Learning
• Data-Level Deep Learning Methods
• Algorithm-Level Deep Learning Techniques
• Hybrid Deep Learning Methods
• Model Calibration

Case Studies

Articles

Videos

Other general resources on the topic

• Libraries, Tools and Frameworks
• Datasets
• Benchmarks
• Awesome libraries on Github
• Books

Oversampling Methods

Technique	Company
Random oversampling	Grab
SMOTE	Microsoft
Borderline-SMOTE	Amazon, Paper

Undersampling Methods

Technique	Company
Negative downsampling	Meta
Negative downsampling	Microsoft
Negative downsampling	Uber

Ensemble Methods

Technique	Company
BalancedBaggingClassifier	Microsoft

Cost-Sensitive Learning

Technique	Company
Cost-sensitive learning	Microsoft
Cost-sensitive learning	Airbnb

Data-Level Deep Learning Methods

Application	Company	Year
Real-Time Personalization	Etsy	2023
Automated Image Tagging	Booking.com	2017
Shot Angle Prediction	Wayfair	2020
Data Protection	Grab	2021
ML Model Improvement	Cloudflare	2022

Algorithm-Level Deep Learning Techniques

Technique	Company
DALL·E 2 pre-training mitigations	OpenAI
BERT for NLP	Wayfair
Focal loss	Meta
Class-balanced loss	Apple

Hybrid Deep Learning Methods

Method	Company	Year
Relational Graph Learning	Uber	2021
Graph for Fraud Detection	Grab	2022

Model Calibration

Technique	Company	Year
Model Calibration	Netflix	2018
Model Calibration	Meta	2014

Case Studies to check out

• Source: ML system design: 300 case studies to learn from

Articles

• Exploring Imbalanced Data Solutions
• Practical Guide to Handle Imbalanced Datasets

Videos

Video Title	Company	Year
Natalie Hockham: Machine learning with imbalanced data sets	GoCardless	2015
PyData Tel Aviv Meetup: GANs for Imbalanced Classification Problems - Moshe Salhov	Playtika	2018
Brendan Herger - Machine Learning Techniques for Class Imbalances & Adversaries	CapitalOne	2016

Other general resources on the topic

Libraries, Tools and Frameworks

• imbalanced-learn
• SMOTE-variants for imbalanced learning
• imbalanced-DL: Deep Imbalanced Learning in Python
• IMBENS: Class-imbalanced Ensemble Learning in Python
• MESA: Meta-sampler for imbalanced learning
• Python-based implementations of algorithms for learning on imbalanced data

Datasets

• CIFAR10-LT and CIFAR100-LT
• Long-tailed Datasets
• Deep Long-Tailed Learning: A Survey: Statistics of Long-Tailed Datasets:

Task	Dataset	# Classes	# Training Data	# Test Data
Image Classification	ImageNet-LT [15]	1,000	115,846	50,000
	CIFAR100-LT [18]	100	50,000	10,000
	Places-LT [15]	365	62,500	36,500
	iNaturalist 2018 [23]	8,142	437,513	24,426
Object Detection/Instance Segmentation	LVIS v0.5 [36]	1,230	57,000	20,000
	LVIS v1 [36]	1,203	100,000	19,800
Multi-label Classification	VOC-LT [37]	20	1,142	4,952
	COCO-LT [37]	80	1,909	5,000
Video Classification	VideoLT [38]	1,004	179,352	51,244

Benchmarks

• Leaderboard: Long-tail Learning on CIFAR-10-LT (ρ=10)
• Leaderboard: Long-tail Learning on CIFAR-100-LT (ρ=100)
• Long-tail Learning Benchmarks

awesome-libraries on Github

• https://github.com/Stomach-ache/awesome-long-tail-learning
• https://github.com/ZhiningLiu1998/awesome-imbalanced-learning/
• https://github.com/CoinCheung/pytorch-loss
• https://github.com/ZhugeKongan/torch-template-for-deep-learning
• https://github.com/hh-xiaohu/Image-augementation-pytorch
• https://github.com/GZWQ/Awesome-Long-Tailed
• https://github.com/danielgy/Paper-list-on-Imbalanced-Time-series-Classification-with-Deep-Learning
• https://gitee.com/liuziyang1106/awesome-imbalanced-learning

Books

• Imbalanced Classification with Python: Better Metrics, Balance Skewed Classes, Cost-Sensitive Learning by by Jason Brownlee
• Learning from Imbalanced Data by Alberto Fernández, Salvador García, Mikel Galar, et al.
• Imbalanced Learning: Foundations, Algorithms, and Applications
• Machine Learning for Imbalanced Data: Tackle imbalanced datasets using machine learning and deep learning techniques

---

Please note that this is a living document and will be updated with more resources over time. Contributions are welcome!