Target_dist labels missing in Target_dist.json

[Victorouledi]

Dear author,

Thank you for the valuable resources you shared.

In the test set you shared in the folder demo_mask, there are some masks for which target labels like following [IMAGE-NAME][MODALITY][SITE]_[TARGET].png are not mentioned in the file Target_dist.json.

For exemple there is no modality and site CT_liver in Target_dist.json and it's not paired with target vessel. But the model is still able to target the vessel for 04_CT_liver.png which matches well with the mask 04_CT_liver_vessel.png.

Is the content of Target_dist.json available on huggingface exhaustive for all the target values the model has been trained on in its latest release ?

[theodore-zhao]

Hi,

The model takes in free text to output segmentation. After the segmentation, it is optional to test if the output belongs to some target type based on K-S test using the statistics in target_dist.json. The current target_dist.json covers most but does not exhaustively include all the modality, site and object pairs the model was trained on. We have added additional object types since the initial release, and if there is any specific one (e.g. vessel in liver CT), please let me know and I can add to the updated file.

[Victorouledi]

Thank you for your answer

Here is a table of what is available in target_dist.json :

<style> </style>

MODALITY_REGION	TARGET
CT-Abdomen	['postcava', 'aorta', 'right kidney', 'kidney', 'left kidney', 'duodenum', 'pancreas', 'liver', 'spleen', 'stomach', 'gallbladder', 'left adrenal gland', 'adrenal gland', 'right adrenal gland', 'esophagus']
CT-Chest	['nodule', 'COVID-19 infection', 'tumor']
MRI-Abdomen	['aorta', 'postcava', 'right kidney', 'duodenum', 'kidney', 'left kidney', 'liver', 'pancreas', 'gallbladder', 'stomach', 'spleen', 'left adrenal gland', 'adrenal gland', 'right adrenal gland', 'esophagus']
MRI-Cardiac	['left heart ventricle', 'myocardium', 'right heart ventricle']
MRI-FLAIR-Brain	['edema', 'tumor core', 'whole tumor']
MRI-T1-Gd-Brain	['enhancing tumor', 'non-enhancing tumor', 'tumor core']
Pathology	['connective tissue cells', 'inflammatory cells', 'neoplastic cells', 'epithelial cells']
X-Ray-Chest	['left lung', 'lung', 'right lung']
Ultrasound-Cardiac	['left heart atrium', 'left heart ventricle']
Endoscopy	['neoplastic polyp', 'polyp', 'non-neoplastic polyp']
Fundus	['optic cup', 'optic disc']
Dermoscopy	['lesion', 'melanoma']
OCT	['edema']

And here is a table of the Modalities + Regions labels associated with their targets that were in demo_mask but not in target_dist.json

<style> </style>

MODALITY_REGION	TARGET
CT_colon	['tumor']
CT_liver	['vessel', 'tumor']
CT_lung	['nodule', 'tumor', 'COVID-19 infection']
CT_pelvis	['uterus']
MRI_heart	['left heart atrium', 'right heart ventricle', 'myocardium', 'left heart ventricle']
MRI_prostate	['prostate']
pathology_colon	['glandular structure']
ultrasound_breast	['tumor']
ultrasound_heart	['left heart ventricle', 'left heart atrium']
ultrasound_transperineal	['fetal head', 'public symphysis']

CT_lung in the first table and CT_chest in the second should be the same thing as for MRI_heart and MRI-Cardiac but there is still 'left heart atrium' missing for MRI-Cardiac.

It would be amazing if you could update target_dist.json

Thank you !

[theodore-zhao]

Thanks for organizing the chart! We have just updated the target_dist.json on HuggingFace with CT-Liver category. While the demo dataset is just to showcase the segmentation capability for different text prompts, this is helpful for us to gradually include more categories for K-S test. If there is a specific type that you would like to prioritize, please let me know.