AdaptSAM is a library built on top of Detectron2 that allows adapting the Segment-Anything Model (SAM) for custom COCO-format datasets. It supports point prompt training and leverages LORA technology for customizable adaptations of SAM.
We provide a script tools/lazyconfig_train_net.py
that trains all configurations of SAM.
To train a model with tools/lazyconfig_train_net.py
, first prepare the datasets following the instructions in
datasets/README.md and then run, for single-node (8-GPUs) NVIDIA-based training:
(node0)$ python ./tools/lazyconfig_train_net.py --config-file configs/finetune/finetune_lora_sam_coco.py --num-gpus 8
To evaluate a trained SAM model's performance, run on single node
(node0)$ python ./tools/lazyconfig_train_net.py --config-file configs/finetune/finetune_lora_sam_coco.py --num-gpus 8 --eval-only --init-from /path/to/checkpoint
To evaluate a original SAM model's performance, run on single node
(node0)$ python ./tools/lazyconfig_train_net.py --config-file configs/eval/eval_sam_coco.py --num-gpus 4 --eval-only
Our environment requirements are consistent with ODISE, for installation, please refer to ODISE
Install dependencies by running:
conda create -n adaptsam python=3.9
conda activate adaptsam
conda install pytorch=1.13.1 torchvision=0.14.1 pytorch-cuda=11.6 -c pytorch -c nvidia
conda install -c "nvidia/label/cuda-11.6.1" libcusolver-dev
pip install -U opencv-python
# under your working directory
git clone https://github.com/facebookresearch/detectron2.git
cd detectron2
pip install -e .
pip install git+https://github.com/cocodataset/panopticapi.git
pip install git+https://github.com/mcordts/cityscapesScripts.git
cd ..
git clone git@github.com:YukunLi99/AdaptSAM.git
cd AdaptSAM
pip install -e .
- Compatible with all features of Detectron2 framework
- Supports custom COCO-format datasets
- Supports point prompt training
- Supports LORA technology
Dataset preparation for AdaptSAM follows Detectron2 and Mask2Former.
For Pascal VOC Instance Segmentation, run the following command to generate coco-format json file (refer to voc2coco):
$ python voc2coco.py \
--ann_dir /path/to/annotation/dir \
--ann_ids /path/to/annotations/ids/list.txt \
--labels /path/to/labels.txt \
--output /path/to/output.json \
method | datasets | |||||
---|---|---|---|---|---|---|
SAM-Base | voc | 1.3 | 34 | 33.8 | 28.4 | 37.5 |
AdaptSAM-Base | voc | 5.8 | 52 | 22.6 | 38.1 | 65.8 |
SAM-Huge | coco | 1.8 | 32.7 | 23.5 | 37.8 | 41.8 |
AdaptSAM-Huge | coco | 10.2 | 49.7 | 28.4 | 59.5 | 73.2 |
Code is largely based on Detectron2, SAM, Mask2Former, ODISE
This project is licensed same as SAM model.