/Zip-Your-CLIP

Primary LanguageJupyter NotebookApache License 2.0Apache-2.0


Zip-Your-CLIP

The Devil is in the Object Boundary: Towards Annotation-free Instance Segmentation using Foundation Models Zip: CLIP Itself is a Good Object-detector ICLR, 2024
Cheng Shi · Sibei Yang†

arXiv PDF


image-20230924124604776

Installation

The code requires python>=3.8, as well as pytorch>=1.7 and torchvision>=0.8. Please follow the instructions here to install both PyTorch and TorchVision dependencies.

Install Segment Anything:

pip install git+https://github.com/facebookresearch/segment-anything.git

Install CLIP and CLIP_Surgery

pip install git+https://github.com/openai/CLIP.git

git clone https://github.com/xmed-lab/CLIP_Surgery.git

Make sure that you correctly install the dependencies of CLIP_Surgery

### Test
device = 'cuda' if torch.cuda.is_available() else 'cpu'
model, preprocess = clip.load("CS-ViT-B/16", device=device)

Getting Started

image-20230924124604777

License

This Project is licensed under the Apache 2.0 license.

Citation

@inproceedings{
shi2024the,
title={The Devil is in the Object Boundary: Towards Annotation-free Instance Segmentation using Foundation Models},
author={Cheng Shi and Sibei Yang},
booktitle={The Twelfth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=4JbrdrHxYy}
}