Deblinder is a visual analytics system for exploring failure reports, descriptions from end-users of how or why a model failed. Deblinder consists of a word embedding visualization for making sense of the reports, and a sidebar for validating hypotheses of systematic failures. To learn more about Deblinder and failure reports, check out the full paper:
Discovering and Validating AI Errors With Crowdsourced Failure Reports
Ángel Alexander Cabrera, Abraham J. Druck, Jason I. Hong, Adam Perer
Proceedings of the ACM on Human-Computer Interaction, Volume 5, Issue CSCW2
- Install Python dependencies with
pip install -r requirements.txt - Install JS dependencies and compile the frontend with
cd client; yarn; yarn build - Run the server with
python app.py
Use the following instructions to use a new dataset and model:
- Follow
preprocessing.ipynbto generate the embedding vectors for a new dataset. - Copy the metadata table with the vectors as
data.csvinclient/public/ - Copy the static images to
img/ - Update the model calls in
app.pyto the new model.