/ltt

Learn then Test: Calibrating Predictive Algorithms to Achieve Risk Control

Primary LanguageJupyter NotebookMIT LicenseMIT

Learn then Test

Open In Colab

This repository will allow you to reproduce the experiments in the Learn then Test paper. For now, please e-mail me if you have trouble reproducing our results.

To install the dependencies for our experiments, run the following command from the root directory of the repository:

conda env create -f environment.yml
conda activate ltt

Then you should be able to run the experiment scripts.

The detectron code is different, and requires a separate set of dependencies. You will also need to make some modifications to the detectron source code.

In the /experiments/detectron folder, please execute

conda env create -f environment.yml
conda activate detectron2

Then you will need to edit the file

~/anaconda3/envs/detectron2/lib/python3.8/site-packages/detectron2/modeling/postprocessing.py

to add the following on line 67 (see the file experiments/detection/postprocessing.py):

results.roi_masks = roi_masks

Then the experiments should run.