[analysis] provide a report of auc vs data modification
Closed this issue · 1 comments
suzannejin commented
- we first fix the data. So for example fix the data to not modified bind set.
- Then compute the AUC of models trained on different data modifications
- Plot test data modification as X, AUC as Y, and color the models based on which data modification is used to train the model.
- Or plot a heatmap with X as blind set data modifications and Y as model modifications, and AUC values. But always with fixed to one kind of data modification per heatmap to have a meaningful analysis
mathysgrapotte commented
cf stimulus-py