ChangeMamba

ChangeMamba: Remote Sensing Change Detection with Spatio-Temporal State Space Model

Hongruixuan Chen^{1 #}, Jian Song^{1,2 #}, Chengxi Han³, Junshi Xia², Naoto Yokoya^{1,2 *}

¹ The University of Tokyo, ² RIKEN AIP, ³ Wuhan University.

^# Equal contribution, ^* Corresponding author

Paper: (arXiv 2404.03425)

Updates

Notice🐍🐍: This repository is being updated! The code and weights for the trained models will be released soon! We'd appreciate it if you could give this repo ⭐️ and stay tuned!
April 13th, 2024: The retrained weights of MambaBCD-Tiny on the LEVIR-CD+ with F1 score of 88.03% are now avaiable. You are welcome to use it!!
April 12th, 2024: The new [arXiv] version containing new accuracy and more experiments is now online! The weights for different models will be released soon!
April 05th, 2024: The [arXiv] version is online!
April 05th, 2024: The models and training code for MambaBCD, MambaSCD, and MambaBDA have been organized and uploaded. You are welcome to use them!!

Overview

ChangeMamba serves as a strong benchmark for change detection tasks, including binary change detection (MambaBCD), semantic change detection (MambaSCD), and building damage assessment (MambaBDA).

Spatio-temporal relationship learning methods of ChangeMamba

Let's Get Started!

A. Installation

The repo is based on the VMama repo, thus you need to install it first. The following installation sequence is taken from the VMamba repo. Also, note that the code in this repo runs under Linux system. We have not tested whether it works under other OS.

Step 1: Clone the repository:

Clone this repository and navigate to the project directory:

git clone https://github.com/ChenHongruixuan/MambaCD.git
cd MambaCD

Step 2: Environment Setup:

It is recommended to set up a conda environment and installing dependencies via pip. Use the following commands to set up your environment:

Create and activate a new conda environment

conda create -n changemamba
conda activate changemamba

Install dependencies

pip install -r requirements.txt
cd kernels/selective_scan && pip install .

Dependencies for Detection and Segmentation (optional in VMamba)

pip install mmengine==0.10.1 mmcv==2.1.0 opencv-python-headless ftfy regex
pip install mmdet==3.3.0 mmsegmentation==1.2.2 mmpretrain==1.2.0

B. Download Pretrained Weight

Also, please download the pretrained weights of VMamba-Tiny, VMamba-Small, and VMamba-Base and put them under

project_path/MambaCD/pretrained_weight/

C. Data Preparation

Binary change detection

The three datasets SYSU, LEVIR-CD+ and WHU-CD are used for binary change detection experiments. Please download them and make them have the following folder/file structure:

${DATASET_ROOT}   # Dataset root directory, for example: /home/username/data/SYSU
├── train
│   ├── T1
│   │   ├──00001.png
│   │   ├──00002.png
│   │   ├──00003.png
│   │   ...
│   │
│   ├── T2
│   │   ├──00001.png
│   │   ... 
│   │
│   └── GT
│       ├──00001.png 
│       ...   
│   
├── test
│   ├── ...
│   ...
│  
├── train.txt   # Data name list, recording all the names of training data
└── test.txt    # Data name list, recording all the names of testing data

Semantic change detection

The SECOND dataset is used for semantic change detection experiments. Please download it and make it have the following folder/file structure:

${DATASET_ROOT}   # Dataset root directory, for example: /home/username/data/SECOND
├── train
│   ├── T1
│   │   ├──00001.png
│   │   ├──00002.png
│   │   ├──00003.png
│   │   ...
│   │
│   ├── T2
│   │   ├──00001.png
│   │   ... 
│   │
│   ├── GT_CD   # Binary change map
│   │   ├──00001.png 
│   │   ... 
│   │
│   ├── GT_T1   # Land-cover map of T1
│   │   ├──00001.png 
│   │   ...  
│   │
│   └── GT_T2   # Land-cover map of T2
│       ├──00001.png 
│       ...  
│   
├── test
│   ├── ...
│   ...
│ 
├── train.txt
└── test.txt

Building damage assessment

The xBD dataset can be downloaded from xView 2 Challenge website. After downloading it, please organize it into the following structure:

${DATASET_ROOT}   # Dataset root directory, for example: /home/username/data/xBD
├── train
│   ├── images
│   │   ├──guatemala-volcano_00000000_pre_disaster.png
│   │   ├──guatemala-volcano_00000000_post_disaster.png
│   │   ...
│   │
│   └── masks
│       ├──guatemala-volcano_00000003_pre_disaster.png
│       ├──guatemala-volcano_00000003_post_disaster.png
│       ... 
│   
├── test
│   ├── ...
│   ...
│
├── holdout
│   ├── ...
│   ...
│
├── train.txt # Data name list, recording all the names of training data
├── test.txt  # Data name list, recording all the names of testing data
└── holdout.txt  # Data name list, recording all the names of holdout data

D. Model Training

Before training models, please enter into [changedetection] folder, which contains all the code for network definitions, training and testing.

cd <project_path>/MambaCD/changedetection

Binary change detection

The following commands show how to train and evaluate MambaBCD-Small on the SYSU dataset:

python script/train_MambaBCD.py  --dataset 'SYSU' \
                                 --batch_size 16 \
                                 --crop_size 256 \
                                 --max_iters 320000 \
                                 --model_type MambaBCD_Small \
                                 --model_param_path '<project_path>/MambaCD/changedetection/saved_models' \ 
                                 --train_dataset_path '<dataset_path>/SYSU/train' \
                                 --train_data_list_path '<dataset_path>/SYSU/train_list.txt' \
                                 --test_dataset_path '<dataset_path>/SYSU/test' \
                                 --test_data_list_path '<dataset_path>/SYSU/test_list.txt'
                                 --cfg '<project_path>/MambaCD/classification/configs/vssm1/vssm_small_224.yaml' \
                                 --pretrained_weight_path '<project_path>/MambaCD/pretrained_weight/vssm_small_0229_ckpt_epoch_222.pth'

Semantic change detection

The following commands show how to train and evaluate MambaSCD-Small on the SECOND dataset:

python script/train_MambaSCD.py  --dataset 'SECOND' \
                                 --batch_size 16 \
                                 --crop_size 256 \
                                 --max_iters 800000 \
                                 --model_type MambaSCD_Small \
                                 --model_param_path '<project_path>/MambaCD/changedetection/saved_models' \ 
                                 --train_dataset_path '<dataset_path>/SECOND/train' \
                                 --train_data_list_path '<dataset_path>/SECOND/train_list.txt' \
                                 --test_dataset_path '<dataset_path>/SECOND/test' \
                                 --test_data_list_path '<dataset_path>/SECOND/test_list.txt'
                                 --cfg '<project_path>/MambaCD/classification/configs/vssm1/vssm_small_224.yaml' \
                                 --pretrained_weight_path '<project_path>/MambaCD/pretrained_weight/vssm_small_0229_ckpt_epoch_222.pth'

Building Damge Assessment

The following commands show how to train and evaluate MambaBDA-Small on the xBD dataset:

python script/train_MambaSCD.py  --dataset 'xBD' \
                                 --batch_size 16 \
                                 --crop_size 256 \
                                 --max_iters 800000 \
                                 --model_type MambaBDA_Small \
                                 --model_param_path '<project_path>/MambaCD/changedetection/saved_models' \ 
                                 --train_dataset_path '<dataset_path>/xBD/train' \
                                 --train_data_list_path '<dataset_path>/xBD/train_list.txt' \
                                 --test_dataset_path '<dataset_path>/xBD/test' \
                                 --test_data_list_path '<dataset_path>/xBD/test_list.txt'
                                 --cfg '<project_path>/MambaCD/classification/configs/vssm1/vssm_small_224.yaml' \
                                 --pretrained_weight_path '<project_path>/MambaCD/pretrained_weight/vssm_small_0229_ckpt_epoch_222.pth'

E. Inference Using Our Weights

Before inference, please enter into [changedetection] folder.

cd <project_path>/MambaCD/changedetection

Binary change detection

The following commands show how to infer binary change maps using trained MambaBCD-Tiny on the LEVIR-CD+ dataset:

python script/infer_MambaBCD.py  --dataset 'LEVIR-CD+' \
                                 --model_type 'MambaBCD_Tiny' \
                                 --test_dataset_path '<dataset_path>/LEVIR-CD+/test' \
                                 --test_data_list_path '<dataset_path>/LEVIR-CD+/test_list.txt' \
                                 --cfg '<project_path>/MambaCD/changedetection/configs/vssm1/vssm_tiny_224_0229flex.yaml' \
                                 --pretrained_weight_path '<project_path>/MambaCD/pretrained_weight/vssm_tiny_0230_ckpt_epoch_262.pth'
                                 --resume '<saved_model_path>/MambaBCD_Tiny_LEVIRCD+_F1_0.8803.pth'

Main Results

The encoders for all the above ChangeMamba models are the the VMamba architecture initialized with ImageNet pre-trained weight.

Binary Change Detection on LEVIR-CD+

Method	Overall Accuracy	F1 Score	IoU	Kappa Coefficient	Param	GFLOPs	ckpts
FC-EF	97.54	70.42	54.34	69.14	17.13	45.74	--
SNUNet	97.83	74.70	59.62	73.57	10.21	176.36	--
DSIFN	98.70	84.07	72.52	83.39	35.73	329.03	--
SiamCRNN-101	98.67	83.20	71.23	82.50	63.44	224.30	--
HANet	98.22	77.56	63.34	76.63	2.61	70.68	--
CGNet	98.63	83.68	71.94	82.97	33.68	329.58	--
TransUNetCD	98.66	83.63	71.86	82.93	28.37	244.54	--
SwinSUNet	98.92	85.60	74.82	84.98	39.28	43.50	--
ChangeFormer V4	98.01	75.87	61.12	74.83	33.61	852.53	--
BIT-101	98.60	82.53	70.26	81.80	17.13	45.74	--
MambaBCD-Tiny	99.03	88.04	78.63	87.53	17.13	45.74	[GDrive][BaiduYun]
MambaBCD-Small	99.02	87.81	78.27	87.30	49.94	114.82	--
MambaBCD-Base	99.06	88.39	79.20	87.91	84.70	179.32	--

Binary Change Detection on WHU-CD

Method	Overall Accuracy	F1 Score	IoU	Kappa Coefficient	Param	GFLOPs	ckpts
FC-EF	98.87	84.89	73.74	84.30	17.13	45.74	--
SNUNet	99.10	87.70	78.09	87.23	10.21	176.36	--
DSIFN	99.31	89.91	81.67	89.56	35.73	329.03	--
SiamCRNN-101	99.19	89.10	80.34	88.68	63.44	224.30	--
HANet	99.16	88.16	78.82	87.72	2.61	70.68	--
CGNet	99.48	92.59	86.21	92.33	33.68	329.58	--
TransUNetCD	99.09	87.79	78.44	87.44	28.37	244.54	--
SwinSUNet	99.50	93.04	87.00	92.78	39.28	43.50	--
ChangeFormer V4	99.10	87.39	77.61	86.93	33.61	852.53	--
BIT-101	99.27	90.04	81.88	89.66	17.13	45.74	--
MambaBCD-Tiny	99.52	93.33	87.49	93.08	17.13	45.74	--
MambaBCD-Small	99.57	94.06	88.79	93.84	49.94	114.82	--
MambaBCD-Base	99.58	94.19	89.02	93.98	84.70	179.32	--

Semantic Change Detection on SECOND

Method	Overall Accuracy	F1 Score	IoU	SeK	Param	GFLOPs	ckpts
HRSCD-S1	45.77	38.44	62.72	5.90	3.36	8.02	--
HRSCD-S2	85.49	49.22	64.43	10.69	6.39	14.29	--
HRSCD-S3	84.62	51.62	66.33	11.97	12.77	42.67	--
HRSCD-S4	86.62	58.21	71.15	18.80	13.71	43.69	--
ChangeMask	86.93	59.74	71.46	19.50	2.97	37.16	--
SSCD-1	87.19	61.22	72.60	21.86	23.39	189.91	--
Bi-SRNet	87.84	62.61	73.41	23.22	19.44	63.72	--
TED	87.39	60.34	72.79	22.17	42.16	75.79	--
SMNet	86.68	60.34	71.95	20.29	19.44	63.72	--
ScanNet	87.86	63.66	73.42	23.94	27.90	264.95	--
MambaSCD-Tiny	88.07	63.44	73.33	23.34	21.51	73.42	--
MambaSCD-Small	88.38	64.10	73.61	24.04	54.28	146.70	--
MambaSCD-Base	88.12	64.03	73.68	24.11	89.99	211.55	--

Building Damage Assessment on xBD

Method	F1_loc	F1_clf	F1_oa	F1_no	F1_minor	F1_major	F1_des	Param	GFLOPs	ckpts
xView2 Baseline	80.47	3.42	26.54	66.31	14.35	0.94	46.57	--	--	--
Siamese-UNet	85.92	65.58	71.68	86.74	50.02	64.43	71.68	--	--	--
MaskRCNN	83.60	70.02	74.10	90.60	49.30	72.20	83.70	--	--	--
ChangeOS-101	85.69	71.14	75.50	89.11	53.11	72.44	80.79	--	--	--
ChangeOS-101-PPS	85.69	75.44	78.52	92.81	59.38	74.65	83.29	--	--	--
DamFormer	86.86	72.81	77.02	89.86	56.78	72.56	80.51	--	--	--
MambaBDA-Tiny	84.76	77.50	79.68	95.33	60.15	75.94	88.27	19.74	59.57	--
MambaBDA-Small	86.61	78.80	81.14	95.99	62.82	76.26	88.37	52.11	130.80	--
MambaBDA-Base	87.38	78.84	81.41	95.94	62.74	76.46	88.58	87.76	195.43	--

Reference

If this code or dataset contributes to your research, please kindly consider citing our paper and give this repo ⭐️ :)

@article{chen2024changemamba,
      title={ChangeMamba: Remote Sensing Change Detection with Spatio-Temporal State Space Model}, 
      author={Hongruixuan Chen and Jian Song and Chengxi Han and Junshi Xia and Naoto Yokoya},
      year={2024},
      eprint={2404.03425},
      archivePrefix={arXiv},
      primaryClass={eess.IV}
}

Acknowledgment

This project is based on VMamba (paper, code), ScanNet (paper, code), thanks for their excellent works.

Q & A

For any questions, please feel free to contact us.

flying318/MambaCD

ChangeMamba

ChangeMamba: Remote Sensing Change Detection with Spatio-Temporal State Space Model

Updates

Overview

Let's Get Started!

A. Installation

B. Download Pretrained Weight

C. Data Preparation

D. Model Training

E. Inference Using Our Weights

Main Results

Binary Change Detection on LEVIR-CD+

Binary Change Detection on WHU-CD

Semantic Change Detection on SECOND

Building Damage Assessment on xBD

Reference

Acknowledgment

Q & A