/PICR-Net_ACMMM23

Primary LanguagePython

PICR-Net_ACMMM2023

Runmin Cong, Hongyu Liu, Chen Zhang*, Wei Zhang, Feng Zheng, Ran Song, and Sam Kwong, Point-aware Interaction and CNN-induced Refinement Network for RGB-D salient object detection, ACM International Conference on Multimedia (ACM MM), 2023.

Network

Our overall framework：

image

Relation Modeling

image

Requirement

Pleasure configure the environment according to the given version:

  • python 3.6.13
  • torch 1.8.2
  • torchvision 0.9.2
  • opencv-python 4.6.0.66
  • numpy 1.19.5
  • timm 0.6.7
  • einops 0.3.2
  • tensorboardx 2.5.1

We also provide ".yaml" files for conda environment configuration, you can download it from [Link], code: mvpl, then use conda env create -f requirement.yaml to create a required environment.

Data Preprocessing

Please follow the tips to download the processed datasets and pre-trained model:

Download RGB-D SOD dataset from [Link], code: mvpl.

Download pretrained backbone weights from [Link], code: mvpl.

├── RGBD_dataset
    ├── train
        ├── RGB
        ├── depth
        ├── GT
    ├── val
        ├── RGB
        ├── depth
        ├── GT
    ├── test
        ├── NJU2K
            ├── RGB
            ├── depth
            ├── GT
        ├── NLPR
            ├── RGB
            ├── depth
            ├── GT
        ...

├── pretrain
    ├── swin_tiny_patch4_window7_224.pth
    ├── vgg16_bn-6c64b313.pth

Training and Testing

Training command :

python train.py

Testing command :

The trained model for PICR-Net can be download here: [Link], code: mvpl.

python test.py

Evaluation

We implement three metrics: MAE (Mean Absolute Error), F-Measure, S-Measure. We use Toolkit [Link] to obtain the test metrics.

Results

  1. Qualitative results: we provide the saliency maps, you can download them from [Link], code: mvpl.
  2. Quantitative results:

image

Bibtex

   @inproceedings{PICR-Net,
     title={Point-aware Interaction and {CNN}-induced Refinement Network for {RGB-D} salient object detection},
     author={Cong, Runmin and Liu, Hongyu and Zhang, Chen and Zhang, Wei and Zheng, Feng and Song, Ran and Kwong, Sam },
     journal={ACM International Conference on Multimedia (ACM MM) },
     year={2023},
    }

Contact Us

If you have any questions, please contact Runmin Cong at rmcong@sdu.edu.cn or Hongyu Liu at liu.hongyu@bjtu.edu.cn.