Eliminating Feature Ambiguity for Few-Shot Segmentation

This repository contains the code for our ECCV 2024 paper "Eliminating Feature Ambiguity for Few-Shot Segmentation", where we design a plug-in network AENet for three existing cross attention-based baselines: CyCTR, SCCAN and HDMNet.

Dependencies

Please follow the official guidelines of the selected baselines to create their virtual environments. Note that:

You can follow SCCAN to create a shared environment for both CyCTR and SCCAN, but remember to run the following commands to install Deformable Attention for CyCTR:
```
> cd SCCAN_CyCTR/model/ops/
> bash make.sh
```
You need to follow HDMNet to create a separate environment for HDMNet.

Directory Structure

The directory structure is:

../
├── SCCAN_CyCTR/  # code for SCCAN and CyCTR
├── HDMNet/       # code for HDMNet
├── lists/        # shared data lists
├── initmodel/    # shared pretrained backbones
└── data/         # shared data
    ├── base_annotation/
    ├── VOCdevkit2012/
    │   └── VOC2012/
    └── MSCOCO2014/           
        ├── annotations/
        ├── train2014/
        └── val2014/

Datasets

PASCAL-5ⁱ: VOC2012 + SBD
COCO-20ⁱ: COCO2014

You can download the pre-processed PASCAL-5ⁱ and COCO-20ⁱ datasets here, and extract them into data/ folder. Then, you need to create a symbolic link to the pascal/VOCdevkit data folder as follows:

> ln -s <absolute_path>/data/pascal/VOCdevkit <absolute_path>/data/VOCdevkit2012

Data Lists

Download the data lists from here and put them into the lists/ directory.
Three baselines share the same data lists, you can put the lists/ folder in the root directory, and create symbolic links as follows:

> ln -s <absolute_path>/lists <absolute_path>/SCCAN_CyCTR/lists
> ln -s <absolute_path>/lists <absolute_path>/HDMNet/lists

Backbones

Download the pretrained backbones from here and put them into the initmodel/ directory.
Three baselines share the same pretrained backbones, you can put the initmodel/ in the root directory, and create symbolic links as follows:

> ln -s <absolute_path>/initmodel <absolute_path>/SCCAN_CyCTR/initmodel
> ln -s <absolute_path>/initmodel <absolute_path>/HDMNet/initmodel

Pretrained Models

Download exp_cyctr.zip to obtain CyCTR-related pretrained models, and extract them to SCCAN_CyCTR/exp.
Download exp_sccan.zip to obtain SCCAN-related pretrained models, and extract them to SCCAN_CyCTR/exp.
Download exp_hdmnet.zip to obtain HDMNet-related pretrained models, and extract them to HDMNet/exp.

Training and Testing Commands

Training (with 4 GPUs)

Take SCCANPlus as an example:

> cd SCCAN_CyCTR

# ============================================================
# Args for train.sh:
#   exp_name: split0/split1/split2/split3 - different folds
#   dataset:  pascal/coco - two benchmark datasets
#   port:     1234 - port for DDP
#   arch:     SCCANPlus/CyCTRPlus - model names
#   net:      vgg/resnet50 - pretrained backbones
#   postfix:  ddp/ddp_5s - 1 or 5 shot
# ============================================================
> sh train.sh split0 pascal 1234 SCCANPlus resnet50 ddp

Testing (with 1 GPUs)