This repo contains the evaluation container for the ISLES 2022 challenge (e.g. ATLAS). The build.sh, test.sh, export.sh scripts along with msot of Dockerfile were generated by evalutils version 0.3.1. The code presented here is not needed by participants for the challenge, but it is made available for transparency and to give participants a way to test the evaluations on their own machines.
The Dockerfile expects ground truth labels to be in the ground-truth directory. The format must be a BIDS dataset
that conforms to the user-supplied settings in settings.py.
The settings file (settings.py) can be used to control data loading, which scoring functions are used, and which
summary statistics are returned.
Adapting this code to other BIDS datasets used in other challenges should require only the following settings:
- Updating
GroundTruthBIDSDerivativeNameandPredictionBIDSDerivativeNameto your derivative names. - Updating
GroundTruthEntitiesandPredictionEntitiesto values matching your data. See the BIDSIO documentation for more information on how to use these. - Importing your desired metrics into
settings.pyand adding them to theScoringFunctionsdictionary.