comic/evalutils

Fixing grand-challenge integration with algorithm/processor template

silvandeleemput opened this issue · 1 comments

Description

The current algorithm/processor template does not work with grand-challenge, because it expects different inputs.

It now seems broken because:

  1. internally MHD files are replaced by MHA files in grand-challenge, and in evalutils MHD files are expected. This can be easily fixed by including
  2. CSV files cannot be served on grand-challenge, because these are not images. This can not be easily solved, so, for now, we'll just make it the example template a candidate detector based on images.

What I Did

  • created the processor/algorithm template with --dev flag (necessary for testing)
  • ran ./build.sh
  • ran ./export.sh
  • uploaded the resulting file to grand-challenge algorithm
  • gave it the test data files (2x*.mhd, 2x*.zraw, 1x*.csv)

For uploaded .mhd/.zraw files:

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/local/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/opt/processor/process.py", line 86, in <module>
    Ddhdemo().process()
  File "/home/processor/.local/lib/python3.7/site-packages/evalutils/evalutils.py", line 138, in process
    self.load()
  File "/home/processor/.local/lib/python3.7/site-packages/evalutils/evalutils.py", line 83, in load
    folder=self._input_path, file_loader=file_loader, filter=filter
  File "/home/processor/.local/lib/python3.7/site-packages/evalutils/evalutils.py", line 115, in _load_cases
    f"Could not load any files in {folder} with " f"{file_loader}."
evalutils.exceptions.FileLoaderError: Could not load any files in /input with <evalutils.io.SimpleITKLoader object at 0x7f8f959fa9d0>.

For uploaded .csv files:

The following image files could not be processed in reader study None:

candidates.csv

The following file formats are supported: .mhd, .mha, .tiff

Note - evalutils should support BOTH mhd/(z)raw and mha.