Implementation of "Segment Anything Model for Medical Images?" in pytorch --for finetuning the SAM with box prompts.
Arxiv link: https://arxiv.org/pdf/2304.14660.pdf
MIA version link: https://www.sciencedirect.com/science/article/pii/S1361841523003213
Json file content.
"Info" refers to the segmentation target in this dataset, while "color" is the ground truth pixel value corresponding to the target.
{
"info": {
"1": "LeftVentricle",
"2": "LeftVentricularMyocardium",
"3": "RightVentricle"
},
"color": {
"1": 85,
"2": 170,
"3": 255
}
}
Dataset distribution.
train: ../data/train_data/images/
val: ../data/train_data/images/
test: ../data/test_data/dataset_name/images/
├── train_data
│ ├── images
│ │ ├── 000001.png
│ │ ├── 000002.png
│ │ └── 000003.png
│ └── labels
│ ├── 00001.png
│ ├── 00002.png
│ └── 00003.png
└── val_data
| ├── images
| │ ├── 000001.png
| │ ├── 000002.png
| │ └── 000003.png
| └── labels
| ├── 00001.png
| ├── 00002.png
| └── 00003.png
└── test_data
├── dataset1
│ ├── images
| | ├── 000001.png
| │ ├── 000002.png
| | └── 000003.png
│ └── labels
| ├── 000001.png
| ├── 000002.png
| └── 000003.png
└── dataset2
├── images
| ├── 000001.png
│ ├── 000002.png
| └── 000003.png
└── labels
├── 000001.png
├── 000002.png
└── 000003.png
$ python pre_grey_rgb2D.py --img_path data/train_data/images --gt_path data/train_data/images --checkpoint sam_vit_b_01ec64.pth #for preparing training data (embeddings) with ViT-B
$ python pre_grey_rgb2D_Huge.py --img_path data/test_data --gt_path data/test_data --task_name 22_Heart --checkpoint sam_vit_b_01ec64.pth #for preparing testing data (embeddings) with ViT-B
$ python train_only_box.py --tr_npz_path data/precompute_vit_b/train --val_npz_path data/precompute_vit_b/valid --model_type vit_b # finetune ViT-B
--tr_npz_path data/precompute_vit_h/train --val_npz_path data/precompute_vit_h/valid --model_type vit_h # finetune ViT-H
$ python test_only_box.py
$ python cal_matric.py
Checkpoints download path: https://drive.google.com/drive/folders/1jry-07RxGYQnT9cQE8weuCZurDCu58pj?usp=sharing
We collected and sorted 53 public datasets to build the large COSMOS 1050K medical image segmentation dataset. Following are the links to the datasets used in our paper.
Ownership and license of the datasets belong to their corresponding original papers, authors, or competition organizers. If you use the datasets, please cite the corresponding paper or links.
Links
https://abdomenct-1k-fully-supervised-learning.grand-challenge.org/
https://www.creatis.insa-lyon.fr/Challenge/acdc/
https://amos22.grand-challenge.org/
https://brainptm-2021.grand-challenge.org/
https://www.med.upenn.edu/cbica/brats2020/data.html
https://www.creatis.insa-lyon.fr/Challenge/camus/databases.html
https://neurips22-cellseg.grand-challenge.org/
https://zenodo.org/record/3431873#.YKIkTfkzbIU
https://blogs.kingston.ac.uk/retinal/chasedb1/
https://www.kaggle.com/datasets/polomarco/chest-ct-segmentation
https://warwick.ac.uk/fac/cross_fac/tia/data/
https://warwick.ac.uk/fac/cross_fac/tia/data/mildnet/
https://crossmoda.grand-challenge.org/
https://polyp.grand-challenge.org/CVCClinicDB/
https://drive.grand-challenge.org/
https://polyp.grand-challenge.org/
https://feta.grand-challenge.org/
https://han-seg2023.grand-challenge.org/
https://amd.grand-challenge.org/
https://palm.grand-challenge.org/
https://idrid.grand-challenge.org/
https://challenge.isic-archive.com/data#2016
https://brain-development.org/ixi-dataset/
https://kipa22.grand-challenge.org/
https://kits19.grand-challenge.org/
https://kits-challenge.org/kits21/
https://datasets.simula.no/kvasir-instrument/
https://datasets.simula.no/kvasir-seg/
https://figshare.com/articles/figure/Left_ventricular_LV_scar_dataset/4214622?file=6875637
https://luna16.grand-challenge.org/
https://www.spinesegmentation-challenge.com/
https://github.com/pangshumao/SpineParseNet
https://www.synapse.org/#!Synapse:syn3193805/wiki/217752
https://promise12.grand-challenge.org/
https://qubiq21.grand-challenge.org/
https://www.kaggle.com/c/siim-acr-pneumothorax-segmentation
https://ski10.grand-challenge.org/
https://sliver07.grand-challenge.org/
https://cecas.clemson.edu/~ahoover/stare/
https://tn-scui2020.grand-challenge.org/
https://github.com/anjany/verse
https://warwick.ac.uk/fac/cross_fac/tia/data/ https://warwick.ac.uk/fac/cross_fac/tia/data/glascontest/download/
https://github.com/HiLab-git/WORD
https://www.epfl.ch/labs/cvlab/data/data-em/
https://imagej.net/events/isbi-2012-segmentation-challenge
https://github.com/wasserth/TotalSegmentator
https://aistudio.baidu.com/aistudio/projectdetail/1952488?channelType=0&channel=0
Our code is based on Segment Anything and MedSAM. We appreciate the authors for their great works. We also sincerely appreciate all the challenge organizers and owners for providing the public medical image segmentation datasets.
If you find the code useful for your research, please cite our paper.
@article{huang2024segment,
title={Segment anything model for medical images?},
author={Huang, Yuhao and Yang, Xin and Liu, Lian and Zhou, Han and Chang, Ao and Zhou, Xinrui and Chen, Rusi and Yu, Junxuan and Chen, Jiongquan and Chen, Chaoyu and others},
journal={Medical Image Analysis},
volume={92},
pages={103061},
year={2024},
publisher={Elsevier}
}