Scribble Hides Class: Promoting Scribble-based Weakly Supervised Semantic Segmentation with Its Class Label

This repository contains training and evaluation codes, and some exmaple images of the dataset. The pretrained checkpoint using resnet50+deeplabV2 is avaliable, which is the used in our ablation study. The complete dataset with distance maps and pseudo-label will be avaliable upon paper acceptance.

Environment setup

anaconda3
- python>=3.8
- ipykernel
- pytorch = 1.13
- numpy
- matplotlib
- torchnet

It is recommanded to in install a anaconda3 for convient environment setup. Run the conda command:

conda env create -f cdsp.yaml

Scripts

The training and evaluating scripts are in scripts/*.sh

Dataset preparation

The complete ScribbleSup dataset can be downloaded google drive. I recollected the ScribbleSup data in 2023 for scribble-supervised semantic segmentation. It is a combination of VOC2012 and SBD, where the relationship may looks like:

ScribbleSup dataset Structure

The original scribble annotations were recorded as a serises of points, where you can find them in ../scribble_annotation/pascal_2012/*.xml. I convert them into the png files with this code by matlab. The data structure:

VOCdevkit/
└── VOC2012
    ├── ImageSets
    ├── pascal_2012_scribble ( I converted scribble points as pngs)
    ├── JPEGImages
    └── SegmentationClassAug

Pseudo label and distance map

Download the pseudo label generated by BMP google drive. Download the distance map of scribble and pseudo label google drive After downloading them, place them under the VOC2012 folder.

(Optional) Make the distance map

The codes for making the distance map is in this repo

Train

Run the example script:

sh scripts/train_r50_deeplabv2.sh

The distance maps and pseudo-label will be avaliable soon.

Eval

Download the pretrained checkpoint from dropbox or googledrive, and move the checkpoint to the log folder, Run the example script:

sh scripts/emlc_r50_deeplabv2.sh

The visualization reults will be saved in emlc_r50_deeplabv2, and the quantity results will be saved in the .txt file.

Demo

Open the demo.ipynb with jupyter notebook, and follow the commands.

Thanks

This repo is inspired by URSS

Zxl19990529/Class-driven-Scribble-Promotion-Network