A curated, but probably biased and incomplete, list of awesome machine learning interpretability resources.
If you want to contribute to this list (and please do!) read over the contribution guidelines, send a pull request, or contact me @jpatrickhall.
An incomplete, imperfect blueprint for a more human-centered, lower-risk machine learning. The resources in this repository can be used to do many of these things today. The resources in this repository should not be considered legal compliance advice.
Image credit: H2O.ai Machine Learning Interpretability team, https://github.com/h2oai/mli-resources.
- Comprehensive Software Examples and Tutorials
- Explainability- or Fairness-Enhancing Software Packages
- Free Books
- Government and Regulatory Documents
- Other Interpretability and Fairness Resources and Lists
- Review and General Papers
- Teaching Resources
- Interpretable ("Whitebox") or Fair Modeling Packages
- AI Incident Tracker
- COMPAS Analysis Using Aequitas
- Explaining Measures of Fairness with SHAP
- Getting a Window into your Black Box Model
- From GLM to GBM Part 1
- From GLM to GBM Part 2
- IML
- Interpretable Machine Learning with Python
- Interpreting Machine Learning Models with the iml Package
- Interpretable Machine Learning using Counterfactuals
- Machine Learning Explainability by Kaggle Learn
- Model Interpretability with DALEX
- Model Interpretation series by Dipanjan (DJ) Sarkar:
- Partial Dependence Plots in R
- Saliency Maps for Deep Learning
- Visualizing ML Models with LIME
- Visualizing and debugging deep convolutional networks
- What does a CNN see?
- acd
- aequitas
- AI Fairness 360
- AI Explainability 360
- ALEPython
- allennlp
- algofairness
- Alibi
- anchor
- BlackBoxAuditing
- casme
- captum
- causalml
- contextual-AI
- ContrastiveExplanation (Foil Trees)
- DeepExplain
- deeplift
- deepvis
- DiCE
- DoWhy
- eli5
- fairml
- fairness-comparison
- fairness_measures_code
- foolbox
- Grad-CAM (GitHub topic)
- iNNvestigate neural nets
- Integrated-Gradients
- interpret_with_rules
- Keras-vis
- keract
- L2X
- lime
- lit
- lofo-importance
- lrp_toolbox
- microsoft/interpret
- MLextend
- ml-fairness-gym
- OptBinning
- parity-fairness
- PDPbox
- pyBreakDown
- PyCEbox
- pymc3
- pytorch-innvestigate
- rationale
- responsibly
- robustness
- RISE
- SALib
- scikit-fairness
- shap
- Skater
- tensorfow/cleverhans
- tensorflow/lucid
- tensorflow/model-analysis
- tensorflow/privacy
- tensorflow/tcav
- tensorfuzz
- TensorWatch
- TextFooler
- tf-explain
- Themis
- themis-ml
- treeinterpreter
- woe
- xai
- xdeep
- yellowbrick
- aif360
- ALEPlot
- breakDown
- DrWhyAI
- DALEX
- DALEXtra
- EloML
- ExplainPrediction
- fastshap
- fairness
- fairmodels
- featureImportance
- forestmodel
- fscaret
- ICEbox
- iml
- lightgbmExplainer
- lime
- live
- mcr
- modelDown
- modelOriented
- modelStudio
- pdp
- shapFlex
- shapleyR
- shapper
- smbinning
- vip
- xgboostExplainer
- An Introduction to Machine Learning Interpretability
- Fairness and Machine Learning
- Interpretable Machine Learning
- 12 CFR Part 1002 - Equal Credit Opportunity Act (Regulation B)
- AI Principles: Recommendations on the Ethical Use of Artificial Intelligence by the Department of Defense
- Algorithmic Accountability Act of 2019
- ALGORITHM CHARTER FOR AOTEAROA NEW ZEALAND
- Artificial Intelligence (AI) in the Securities Industry
- Article 22 EU GDPR
- Assessment List for Trustworthy Artificial Intelligence (ALTAI) for self-assessment - Shaping Europe’s digital future - European Commission
- Audit of Governance and Protection of Department of Defense Artificial Intelligence Data and Technology
- A Primer on Artificial Intelligence in Securities Markets
- Booker Wyden Health Care Letters
- California Consumer Privacy Act (CCPA)
- Consultation on the OPC’s Proposals for ensuring appropriate regulation of artificial intelligence
- Directive on Automated Decision Making
- Facial Recognition and Biometric Technology Moratorium Act of 2020
- General principles for the use of Artificial Intelligence in the financial sector
- Gouvernance des algorithmes d’intelligence artificielle dans le secteur financier (French)
- Innovation spotlight: Providing adverse action notices when using AI/ML models
- Office of Management and Budget Draft Guidance for Regulation of Artificial Intelligence Applications
- On Artificial Intelligence - A European approach to excellence and trust
- Opinion of the German Data Ethics Commission
- Principles of Artificial Intelligence Ethics for the Intelligence Community
- Questions and Answers to Clarify and Provide a Common Interpretation of the Uniform Guidelines on Employee Selection Procedures
- RE: Use of External Consumer Data and Information Sources in Underwriting for Life Insurance
- Singapore Personal Data Protection Commission (PDPC) Model Artificial Intelligence Governance Framework
- SUPERVISORY GUIDANCE ON MODEL RISK MANAGEMENT
- U.K. Information Commissioner's Office (ICO) AI Audting Framework (overview series)
- U.S FDA Proposed Regulatory Framework for Modifications to Artificial Intelligence/Machine Learning (AI/ML)-Based Software as a Medical Device (SaMD)
- Using Artificial Intelligence and Algorithms
- 8 Principles of Responsible ML
- ACM FAT* 2019 Youtube Playlist
- AI Ethics Guidelines Global Inventory
- AI Incident Database
- AllenNLP Interpret: A Framework for Explaining Predictions of NLP Models
- Algorithms and prejudice
- Awesome interpretable machine learning ;)
- Awesome machine learning operations
- Awful AI
- algoaware
- BIML Interactive Machine Learning Risk Framework
- Beyond Explainability: A Practical Guide to Managing Risk in Machine Learning Models
- criticalML
- Debugging Machine Learning Models (ICLR workshop proceedings)
- Decision Points in AI Governance
- Deep Insights into Explainability and Interpretability of Machine Learning Algorithms and Applications to Risk Management
- Distill
- Fairness, Accountability, and Transparency in Machine Learning (FAT/ML) Scholarship
- From Principles to Practice: An interdisciplinary framework to operationalise AI ethics
- How will the GDPR impact machine learning?
- Machine Learning Ethics References
- Machine Learning Interpretability Resources
- Machine Learning: Considerations for fairly and transparently expanding access to credit
- MIT AI Ethics Reading Group
- On the Responsibility of Technologists: A Prologue and Primer
- private-ai-resources
- Problems with Shapley-value-based explanations as feature importance measures
- Real-World Model Debugging Strategies
- ResponsibleAI
- Sample AI Incident Response Checklist
- Ten Questions on AI Risk
- Testing and Debugging in Machine Learning
- Troubleshooting Deep Neural Networks
- Warning Signs: The Future of Privacy and Security in an Age of Machine Learning
- XAI Resources
- You Created A Machine Learning Application Now Make Sure It's Secure
- 50 Years of Test (Un)fairness: Lessons for Machine Learning
- A Comparative Study of Fairness-Enhancing Interventions in Machine Learning
- A Survey Of Methods For Explaining Black Box Models
- A Marauder’s Map of Security and Privacy in Machine Learning
- Challenges for Transparency
- Closing the AI Accountability Gap
- Explaining Explanations: An Overview of Interpretability of Machine Learning
- Explanation in Human-AI Systems: A Literature Meta-Review, Synopsis of Key Ideas and Publications, and Bibliography for Explainable AI
- Interpretable Machine Learning: Definitions, Methods, and Applications
- Limitations of Interpretable Machine Learning
- Machine Learning Explainability in Finance
- On the Art and Science of Machine Learning Explanations
- Please Stop Explaining Black Box Models for High-Stakes Decisions
- The Mythos of Model Interpretability
- Towards A Rigorous Science of Interpretable Machine Learning
- Toward Trustworthy AI Development: Mechanisms for Supporting Verifiable Claims
- The Security of Machine Learning
- Techniques for Interpretable Machine Learning
- Trends and Trajectories for Explainable, Accountable and Intelligible Systems: An HCI Research Agenda
- An Introduction to Data Ethics
- Fairness in Machine Learning
- Fast.ai Data Ethics course
- Human-Center Machine Learning
- Introduction to Responsible Machine Learning
- Trustworthy Deep Learning
- Bayesian Case Model
- Bayesian Ors-Of-Ands
- Bayesian Rule List (BRL)
- Explainable Boosting Machine (EBM)/GA2M
- fair-classification
- Falling Rule List (FRL)
- H2O-3
- Optimal Sparse Decision Trees
- Monotonic XGBoost
- Multilayer Logical Perceptron (MLLP)
- pyGAM
- pySS3
- Risk-SLIM
- Scikit-learn
- sklearn-expertsys
- skope-rules
- Super-sparse Linear Integer models (SLIMs)
- tensorflow/lattice
- This Looks Like That
- arules
- Causal SVM
- elasticnet
- gam
- glm2
- glmnet
- H2O-3
- Monotonic XGBoost
- quantreg
- rpart
- RuleFit
- Scalable Bayesian Rule Lists (SBRL)
- Mar 1988 - A blot on the profession
- Jul 2015 - Google says sorry for racist auto-tag in photo app
- Mar 2016 - Here Are the Microsoft Twitter Bot’s Craziest Racist Rants
- Jun 2016 - Google faulted for racial bias in image search results for black teenagers
- Oct 2016 - 'Rogue' Algorithm Blamed for Historic Crash of the British Pound
- Oct 2016 - Crime-prediction tool PredPol amplifies racially biased policing, study shows
- May 2017 - Houston Schools Must Face Teacher Evaluation Lawsuit
- Jun 2017 - When a Computer Program Keeps You in Jail
- Feb 2018 - Study finds gender and skin-type bias in commercial artificial-intelligence systems
- Mar 2018 - Self-Driving Uber Car Kills Pedestrian in Arizona, Where Robots Roam
- Mar 2018 - AI-Assisted Fake Porn Is Here and We're All F***ed
- Jul 2018 - Amazon’s Face Recognition Falsely Matched 28 Members of Congress With Mugshots
- Oct 2018 - Amazon scraps 'sexist AI' recruiting tool that showed bias against women
- Dec 2018 - AI start-up that scanned babysitters halts launch following Post Report
- Jan 2019 - Cambridge Analytica’s parent pleads guilty to breaking UK data law
- May 2019 - Investor Sues After an AI’s Automated Trades Cost Him $20 Million
- Jun 2019 - Google and the University of Chicago Are Sued Over Data Sharing
- Sep 2019 - The viral selfie app ImageNet Roulette seemed fun – until it called me a racist slur
- Sep 2019 - Scammer Successfully Deepfaked CEO's Voice To Fool Underling Into Transferring $243,000
- Oct 2019 - Oh dear... AI models used to flag hate speech online are, er, racist against black people
- Oct 2019 - Dissecting racial bias in an algorithm used to manage the health of populations
- Nov 2019 - NY regulator investigating Apple Card for possible gender bias
- Nov 2019 - Chinese-style facial recognition technology is trialled in Australian schools to register pupils - sparking major privacy concerns
- Dec 2019 - Tenants sounded the alarm on facial recognition in their buildings. Lawmakers are listening
- Dec 2019 - Researchers bypass airport and payment facial recognition systems using masks
- Feb 2020 - An Indian politician is using deepfake technology to win new voters
- Feb 2020 - Suckers List: How Allstate’s Secret Auto Insurance Algorithm Squeezes Big Spenders
- Feb 2020 - Tesla Autopilot gets tricked into accelerating from 35 to 85 mph with modified speed limit sign
- Mar 2020 - Netherlands: Court Prohibits Government’s Use of AI Software to Detect Welfare Fraud
- Apr 2020 - Google apologizes after its Vision AI produced racist results
- May 2020 - Researchers find major demographic differences in speech recognition accuracy
- May 2020 - Access Denied: Faulty Automated Background Checks Freeze Out Renters
- May 2020 - A.C.L.U. Accuses Clearview AI of Privacy ‘Nightmare Scenario’
- May 2020 - Walmart Employees Are Out to Show Its Anti-Theft AI Doesn't Work
- May 2020 - Robodebt removed humans from Human Services, and the Government is facing the consequences
- Jun 2020 - Government’s Use of Algorithm Serves Up False Fraud Charges
- Jun 2020 - Microsoft's robot editor confuses mixed-race Little Mix singers
- Jun 2020 - Tweet: "This algorithm probably made this mistake ..." (President Obama de-blurred into white male)
- Jun 2020 - Detroit Police Chief: Facial Recognition Software Misidentifies 96% of the Time
- Jun 2020 - Wrongfully Accused by an Algorithm
- Jun 2020 - An Algorithm that "Predicts" Criminality Based on a Face Sparks a Furor
- Jun 2020 - PwC facial recognition tool criticised for home working privacy invasion
- Jul 2020 - ISIS 'still evading detection on Facebook', report says
- Jul 2020 - Meet the Secret Algorithm That's Keeping Students Out of College
- Jul 2020 - Rite Aid deployed facial recognition systems in hundreds of U.S. stores
- Jul 2020 - Tweet: "Oh, dear ..." (GPT-3 anti-semitism)
- Jul 2020 - Google Ad Portal Equated “Black Girls” with Porn
- Jul 2020 - Facial biometrics training dataset leads to BIPA lawsuits against Amazon, Alphabet and Microsoft
- Aug 2020 - Police use of facial recognition unlawfully breached privacy rights, says Court of Appeal ruling
- Aug 2020 - There is nothing 'fair' about judging A-levels by algorithm
- Aug 2020 - When algorithms define kids by postcode: UK exam results chaos reveal too much reliance on data analytics
- Aug 2020 - Macy’s hit with privacy lawsuit over alleged use of controversial facial recognition software
- Aug 2020 - Google’s Advertising Platform Is Blocking Articles About Racism
- Aug 2020 - Home Office drops 'racist' algorithm from visa decisions
- Aug 2020 - De Blasio Will Reassess NYPD's Use Of Facial Recognition Tech After Protester Arrest
- Aug 2020 - Facebook algorithm recommending Holocaust denial and fascist content, report finds
- Aug 2020 - Report: AI Company Leaks Over 2.5M Medical Records
- Aug 2020 - Watchdog investigates Barclays for spying on staff