MUSTER: A Multi-scale Transformer-based Decoder for Semantic Segmentation
The core codes can be found at:
MUSTER/mmseg/models/decode_heads/MUSTER_head.py
We use MMSegmentation v0.29.0 as the codebase.
For install , please refer to the guidelines in MMSegmentation v0.29.0.
An example (works for me): CUDA 11.6
and pytorch 1.12.1
Step 0. Install MMCV using MIM.
pip install -U openmim
mim install mmcv-full
Step 1. Install MUSTER.
# The workdir of the terminal should be MUSTER/
pip install -v -e .
# "-v" means verbose, or more output
# "-e" means installing a project in editable mode,
# thus any local modifications made to the code will take effect without reinstallation.
For dataset preparation, please refer to the guidelines in this link.
It is recommended to symlink the dataset root to MUSTER/data
.
If your folder structure is different, you may need to change the corresponding paths in config files.
The fold structure is recommended to be:
MUSTER
├── data
│ ├── cityscapes
│ │ ├── leftImg8bit
│ │ │ ├── train
│ │ │ ├── val
│ │ ├── gtFine
│ │ │ ├── train
│ │ │ ├── val
│ ├── ade
│ │ ├── ADEChallengeData2016
│ │ │ ├── annotations
│ │ │ │ ├── training
│ │ │ │ ├── validation
│ │ │ ├── images
│ │ │ │ ├── training
│ │ │ │ ├── validation
The data could be found here after registration.
By convention, **labelTrainIds.png
are used for cityscapes training.
MMsegmentation provided a script based on cityscapesscripts
to generate **labelTrainIds.png
.
# --nproc means 8 process for conversion, which could be omitted as well.
python tools/convert_datasets/cityscapes.py data/cityscapes --nproc 8
The training and validation set of ADE20K could be download from this link. You may also download test set from here.
Because of anonymity, we cannot give trained weights at present.
Example: evaluate MUSTER
with Swin Transformer
on ADE20K
:
# Single-gpu testing
python tools/test.py configs/MUSTER/MUSTER_swin-base_ade20k_160k.py /path/to/checkpoint_file --show
Example: evaluate MUSTER
with Swin Transformer
on Cityscapes
:
# Single-gpu testing
python tools/test.py configs/MUSTER/MUSTER_swin-base_ade20k_160k.py /path/to/checkpoint_file --show
Example: train MUSTER
with Swin Transformer
on ADE20K
:
# Single-gpu training
python tools/train.py configs/MUSTER/MUSTER_swin-base_ade20k_160k.py
Please check the LICENSE file.
Please check CITATION.cff
.