[IMP] Extend SeriesToVolume Operator to support multi-channel data
jtetrea opened this issue · 3 comments
Is your enhancement request related to a problem? Please describe.
For certain imaging problems, multiple MRI modalities provide complimentary information and can be used together in a multi-channel image as input to an AI model. Right now, only a single series can be processed at a time, limiting the AI processing to a single channel.
Describe the solution you'd like
The solution to this is to extend the SeriesToVolume operator to ingest multiple selected series and package them properly together into an Image object before passing them to an inference operator.
Describe alternatives you've considered
Any alternatives I've considered require more effort to change multiple operators in order to handle this type of data. SeriesToVolume being extended would make sure that any downstream operator sees this output as a canonical MONAI Image and shouldn't create any more disruptions.
Additional context
Will be extending initial version to handle 3-series (e.g. T2, ADC, HighB) input. May additionally extend beyond this to handle n-series.
I think having an atomic SeriesToVolumeOperator
is a good idea. I suggest an alternative which is a MergeVolumeOperator
. The idea will be to use multiple SeriesToVolumeOperator
and then use the MergeVolumeOperator
to build a multi-channel image. What do you think?
I think having an atomic
SeriesToVolumeOperator
is a good idea. I suggest an alternative which is aMergeVolumeOperator
. The idea will be to use multipleSeriesToVolumeOperator
and then use theMergeVolumeOperator
to build a multi-channel image. What do you think?
Assuming the dims, pixel spacing and affine are the same among all the candidate images, the proposed MergeVolumeOperator
will simply stack them up as multi-channel image. I think make sure the conditions are satisfied by loading and adjusting pixel data from multiple DICOM series is the difficult part and is solved by the multi-series to vol operator. There might be pixel value normalization across all series of a specific modality.
There are definitely other use cases for the multi series to single image loader, e.g. the BRATS Brain Tumor app.
Addressed by MergeVolumeOperator in branch JT/MergeVolume.