This repository contains the source code of PRCL from the paper, Boosting Pixel-Wise Contrastive Learning with Probabilitic Representations, proposed by Haoyu Xie, Changqi Wang, Mingkai Zheng, Minjing Dong, Shan You, Chong Fu, and Chang Xu. The paper is accepted to AAAI 2023.
Nov. 2022 -- Upload the sorce code.
PRCL is evaluated with two datasets: PASCAL VOC 2012 and CityScapes.
- For PASCAL VOC, please download the original training images from the official PASCAL site:
VOCtrainval_11-May-2012.tar
and the augmented labels here:SegmentationClassAug.zip
. Extract the folderJPEGImages
andSegmentationClassAug
as follows:
├── data
│ ├── VOCdevkit
│ │ ├──VOC2012
│ │ | ├──JPEGImages
│ │ | ├──SegmentationClassAug
│ │ | ├──prefix
│ │ | | ├──val.txt
│ │ | | ├──train_aug.txt
- For CityScapes, please download the original images and labels from the official CityScapes site:
leftImg8bit_trainvaltest.zip
andgtFine_trainvaltest.zip
. Extract the folderleftImg8bit_trainvaltest.zip
andgtFine_trainvaltest.zip
as follows:
├── data
│ ├── cityscapes
│ │ ├──leftImg8bit
│ │ | ├──train
│ │ | ├──val
│ │ ├──train
│ │ ├──val
Folders train
and val
under leftImg8bit
contains training and validation images while folders train
and val
under leftImg8bit
contains labels.
PRCL uses ResNet-101 pretrained on ImageNet, please download from here and change the direction in corresponding python file.
In order to install the correct environment, please run the following script:
conda create -n prcl python=3.8.5
conda activate prcl
pip install -r requirements.txt
It may takes a long time, take a break and have a cup of coffee! It is OK if you want to install environment manually, remember to check CAREFULLY!
You can run our code with a single GPU or multiple GPUs.
- For single GPU users, please run prcl_sig.py
- For multiple GPUs users, please run the following script:
run ./script/batch_train.sh
All hyper-parameters used in the code are shown below:
Name | Discription | Value |
---|---|---|
alpha |
hyper-parameter in EMA model | 0.99 |
lr |
learning rate of backbone, prediction head, and project head | 3.2e-3 |
uncer_lr |
learning rate of probability head | 5e-5 |
uncer_lr |
learning rate of probability head | 5e-5 |
un_threshold |
threshold in unsupervised loss | 0.97 |
weak_threshold |
weak threshold in PRCL loss | 0.7 |
strong_threshold |
strong threshold in PRCL loss | 0.8 |
temp |
temperature in PRCL loss | 100 |
num_queries |
number of queries in PRCL loss | 256 |
num_negatives |
number of negatives in PRCL loss | 512 |
begin_epoch |
the begin epoch of scheduler |
0 |
max_epoch |
the end epoch of scheduler |
200 |
max_value |
the max value of scheduler |
1.0 |
min_value |
the min value of scheduler |
0 |
ramp_mult |
the \alpha of scheduler |
-5.0 |
The data processing and augmentation (CutMix, CutOut, and ClassMix) are borrowed from ReCo.
Thanks a lot for their splendid work!
If you think this work is useful for you and your research, please considering citing the following:
@article{PRCL,
title={Boosting Semi-Supervised Semantic Segmentation with Probabilistic Representations},
author={Xie, Haoyu and Wang, Changqi and Zheng, Mingkai and Dong, Minjing and You, Shan and Xu, Chang},
journal={arXiv preprint arXiv:2210.14670},
year={2022}
}
If you have any questions or meet any problems, please feel free to contact us.
- Haoyu Xie, 895852154@qq.com
- Changqi Wang, wangchangqi98@gmail.com