/Clip-Seg

Open-Text Semantics Segmentation for Robotics Manipulation

Primary LanguagePythonMIT LicenseMIT

Clip-Seg

MIT 6.843 Course Project. We use a simple idea for open-text semantic segmentation task. The code largely follows Cliport and Uois. We hope this code can be used for robotic researchers that require good generalization property in language and vision-wise segmentation.

Slide Report

image

Installation

git clone https://github.com/liruiw/Clip-Seg.git --recursive

  1. Setup: Ubuntu 16.04 or above, CUDA 10.0 or above, python 3.6

    • Install Clip-Seg inside a new conda environment 
      conda create --name clip-seg python=3.7
conda activate clip-seg
pip install -r requirements.txt

  3. Download Dataset OCID, OCID-Ref, TOD dataset

  4. Download pretrained models bash download_model.sh

  5. Example Scripts:

Test TOD python3 test_clipseg.py --model_suffix 100x100 --config_name full_TOD

Train OCID python3 train_clipseg.py --config_name ocid

Demo OCID python3 demo_clipseg.py --config_name ocid

Could be combined with learned policy for picking demo robot_clipseg.py

Note

  1. Please use Github issue tracker to report bugs. For other questions please contact Lirui Wang.
  2. The Clip-Seg is licensed under the MIT License.