Scale-aware Deep Reinforcement Learning for High Resolution Remote Sensing Imagery Classification

Tired of being restricted by limited network input sizes (e.g., 512x512) in high-resolution image segmentation? Let the network choose its own scale!

Getting Started

This repository contains the implementation of our project using stable-baselines3==1.6 and gym==0.21. To ensure compatibility and optimal performance, please use these specific versions.

For users employing stable-baselines3>=1.7, adjustments to the Environment may be required for compatibility with gymnasium. Refer to the documentation of stable-baselines3 and gymnasium for guidance.

Model Download

The pre-trained agent can be found in Goole Drive.

Structure of model.zip:

saved_model.zip/
├── data.json          - JSON file containing class parameters (dictionary format)
├── *.optimizer.pth    - Serialized PyTorch optimizers
├── policy.pth         - PyTorch state dictionary for the saved policy
├── pytorch_variables.pth - Additional PyTorch variables
├── version.txt        - Stable Baselines3 version used for model saving
├── system_info.txt    - System information (OS, Python version, etc.) during model saving

Datasets

Datasets used in our research:

Gaofen Image Dataset (GID):
- Download
- Reference
Five-Billion-Pixels (FBP) Dataset:
- Download
- Reference
Wuhan Urban Semantic Understanding (WUSU) Dataset:
- Download
- Reference

Baseline & Comparative Methods

We primarily utilize the segmentation_models_pytorch library.

The methods we compare include:

Global-Local Networks (GLNet): Code | Paper
CascadePSP: Code | Paper
Progressive Semantic Segmentation (MagNet): Code | Paper
Wider Context Transformer (WiCoNet): Code | Paper

Usage

For step-by-step instructions, please refer to the Example.ipynb notebook in this repository.

The agent is trained on images approximately 6000x7000 in size. While it can be tested on images of any size, please note that results may vary.

Issues and Contributions

Encounter an issue or have a question? Feel free to open an issue in this repository.

Citation

If you use our Scale Control Agent in your research, please cite our paper:

@article{LIU2024296,
title = {Scale-aware deep reinforcement learning for high resolution remote sensing imagery classification},
journal = {ISPRS Journal of Photogrammetry and Remote Sensing},
volume = {209},
pages = {296-311},
year = {2024},
issn = {0924-2716},
doi = {https://doi.org/10.1016/j.isprsjprs.2024.01.013},
url = {https://www.sciencedirect.com/science/article/pii/S0924271624000224},
author = {Yinhe Liu and Yanfei Zhong and Sunan Shi and Liangpei Zhang},
}

Additional Resources

Discover more resources and datasets from our group on our website.

Commercial use is prohibited.

For further inquiries, contact me at liuyinhe@whu.edu.cn.

fyan1024/ScaleControlAgent