ShipRSImagaeNet: A Large-scale Fine-Grained Dataset for Ship Detection in High-Resolution Optical Remote Sensing Images
Description
ShipRSImageNet is a large-scale fine-grainted dataset for ship detection in high-resolution optical remote sensing images. The dataset contains 3,435 images from various sensors, satellite platforms, locations, and seasons. Each image is around 930×930 pixels and contains ships with different scales, orientations, and aspect ratios. The images are annotated by experts in satellite image interpretation, categorized into 50 object categories images. The fully annotated ShipRSImageNet contains 17,573 ship instances. There are five critical contributions of the proposed ShipRSImageNet dataset compared with other existing remote sensing image datasets.
-
Images are collected from various remote sensors cover- ing multiple ports worldwide and have large variations in size, spatial resolution, image quality, orientation, and environment.
-
Ships are hierarchically classified into four levels and 50 ship categories.
-
The number of images, ship instances, and ship cate- gories is larger than that in other publicly available ship datasets. Besides, the number is still increasing.
-
We simultaneously use both horizontal and oriented bounding boxes, and polygons to annotate images, providing detailed information about direction, background, sea environment, and location of targets.
-
We have benchmarked several state-of-the-art object detection algorithms on ShipRSImageNet, which can be used as a baseline for future ship detection methods.
Examples of Annotated Images
Image Source and Usage License
The ShipRSImageNet dataset collects images from a variety of sensor platforms and datasets, in particular:
-
Images of the xView dataset are collected from WorldView-3 satellites with 0.3m ground resolution. Images in xView are pulled from a wide range of geographic locations. We only extract images with ship targets from them. Since the image in xView is huge for training, we slice them into 930×930 pixels with 150 pixels overlap to produce 532 images and relabeled them with both horizontal bounding box and oriented bounding box.
-
We also collect 1,057 images from HRSC2016 and 1,846 images from FGSD datasets, corrected the mislabeled and relabeled missed small ship targets.
-
21 images from the Airbus Ship Detection Challenge.
-
17 images from Chinese satellites suchas GaoFen-2 and JiLin-1.
Use of the Google Earth images must respect the "Google Earth" terms of use.
All images and their associated annotations in ShipRSImageNet can be used for academic purposes only, but any commercial use is prohibited.
Object Category
The ship classification tree of proposed ShipRSImageNet is shown in the following figure. Level 0 distinguish whether the object is a ship, namely Class. Level 1 further classifies the ship object category, named as Category. Level 2 further subdivides the categories based on Level 1. Level 3 is the specific type of ship, named as Type.
At Level 3, ship objects are divided into 50 types. For brevity, we use the following abbreviations: DD for Destroyer, FF for Frigate, LL for Landing, AS for Auxiliary Ship, LSD for Landing Ship Dock, LHA for Landing Heli- copter Assault Ship, AOE for Fast Combat Support Ship, EPF for Expeditionary Fast Transport Ship, and RoRo for Roll- on Roll-off Ship. These 50 object classes are Other Ship, Other Warship, Submarine, Other Aircraft Carrier, Enterprise, Nimitz, Midway, Ticonderoga, Other Destroyer, Atago DD, Arleigh Burke DD, Hatsuyuki DD, Hyuga DD, Asagiri DD, Other Frigate, Perry FF, Patrol, Other Landing, YuTing LL, YuDeng LL, YuDao LL, YuZhao LL, Austin LL, Osumi LL, Wasp LL, LSD 41 LL, LHA LL, Commander, Other Auxiliary Ship, Medical Ship, Test Ship, Training Ship, AOE, Masyuu AS, Sanantonio AS, EPF, Other Merchant, Container Ship, RoRo, Cargo, Barge, Tugboat, Ferry, Yacht, Sailboat, Fishing Vessel, Oil Tanker, Hovercraft, Motorboat, and Dock.
Dataset Download
- Baidu Drive (Extraction code:h2qk):
- Google Drive:
Benchmark Code Installation
We keep all the experiment settings and hyper-parameters the same as depicted in MMDetection(v2.11.0) config files except for the number of categories and parameters. MMDe- tection is an open-source object detection toolbox based on PyTorch. It is a part of the OpenMMLab project developed by Multimedia Laboratory, CUHK.
This project is based on MMdetection(v2.11.0). MMDetection is an open source object detection toolbox based on PyTorch. It is a part of the OpenMMLab project.
Prerequisites
- Linux or macOS (Windows is in experimental support)
- Python 3.6+
- PyTorch 1.3+
- CUDA 9.2+ (If you build PyTorch from source, CUDA 9.0 is also compatible)
- GCC 5+
- MMCV
Installation
-
Install MMdetection following the instructions. We are noting that our code is checked in mmdetection V2.11.0 and pytorch V1.7.1.
-
Create a conda virtual environment and activate it.
conda create -n open-mmlab python=3.7 -y conda activate open-mmlab
-
Install PyTorch and torchvision following the official instructions, e.g.,
conda install pytorch torchvision -c pytorch
Note: Make sure that your compilation CUDA version and runtime CUDA version match. You can check the supported CUDA version for precompiled packages on the PyTorch website.
-
Install mmcv-full, we recommend you to install the pre-build package as below.
pip install mmcv-full -f https://download.openmmlab.com/mmcv/dist/{cu_version}/{torch_version}/index.html
Please replace
{cu_version}and{torch_version}in the url to your desired one. For example, to install the latestmmcv-fullwithCUDA 11andPyTorch 1.7.1, use the following command:pip install mmcv-full -f https://download.openmmlab.com/mmcv/dist/cu110/torch1.7.1/index.html
-
Download this benchmark code.
git clone https://github.com/open-mmlab/mmdetection.git cd mmdetection2.11-ShipRSImageNet
-
Install build requirements and then install MMDetection.
pip install -r requirements/build.txt pip install -v -e . # or "python setup.py develop"
-
Train with ShipRSImageNet
-
Download the ShipRSImageNet dataset. It is recommended to symlink the ShipRSImageNet dataset root to $mmdetection2.11-ShipRSImageNet/data:
ln -s $dataset/ShipRSImageNet/ $mmdetection2.11-ShipRSImageNet/data/
-
If your folder structure is different, you may need to change the corresponding paths in config files.
-
mmdetection2.11-ShipRSImageNet ├── mmdet ├── tools ├── configs ├── data │ ├── ShipRSImageNet │ │ ├── COCO_Format │ │ ├── masks │ │ ├── VOC_Format │ │ │ ├── annotations │ │ │ ├── ImageSets │ │ │ ├── JPEGImages
-
Prepare a config file:
-
The benchamark config file of ShipRSImageNet already in the following:
-
$mmdetection2.11-ShipRSImageNet/configs/ShipRSImageNet/
-
-
-
Example of train a model with ShipRSImageNet:
-
python tools/train.py configs/ShipRSImageNet/faster_rcnn/faster_rcnn_r50_fpn_100e_ShipRSImageNet_Level0.py
-
Models trained on ShipRSImageNet
We introduce two tasks: detection with horizontal bounding boxes (HBB for short) and segmentation with oriented bounding boxes (SBB for short). HBB aims at extracting bounding boxes with the same orientation of the image, it is an Object Detection task. SBB aims at semantically segmenting the image, it is a Semantic Segmentation task.
The evaluation protocol follows the same mAP and mAR of area small/medium/large and mAP(@IoU=0.50:0.95) calculation used by MS-COCO.
Level 0
| Model | Backbone | Style | HBB mAP | SBB mAP | Extraction code | Download |
|---|---|---|---|---|---|---|
| Faster RCNN with FPN | R-50 | Pytorch | 0.550 | 2vrm | model | |
| Faster RCNN with FPN | R-101 | Pytorch | 0.546 | f362 | model | |
| Mask RCNN with FPN | R-50 | Pytorch | 0.566 | 0.440 | 24eq | model |
| Mask RCNN with FPN | R-101 | Pytorch | 0.557 | 0.436 | lbcb | model |
| Cascade Mask RCNN with FPN | R-50 | Pytorch | 0.568 | 0.430 | et6m | model |
| SSD | VGG16 | Pytorch | 0.464 | qabf | model | |
| Retinanet with FPN | R-50 | Pytorch | 0.418 | 7qdw | model | |
| Retinanet with FPN | R-101 | Pytorch | 0.419 | vdiq | model | |
| FoveaBox | R-101 | Pytorch | 0.453 | urbf | model | |
| FCOS with FPN | R-101 | Pytorch | 0.333 | 94ub | model | |
Level 1
| Model | Backbone | Style | HBB mAP | SBB mAP | Extraction code | Download |
|---|---|---|---|---|---|---|
| Faster RCNN with FPN | R-50 | Pytorch | 0.366 | - | 5i5a | model |
| Faster RCNN with FPN | R-101 | Pytorch | 0.461 | - | 6ts7 | model |
| Mask RCNN with FPN | R-50 | Pytorch | 0.456 | 0.347 | 9gnt | model |
| Mask RCNN with FPN | R-101 | Pytorch | 0.472 | 0.371 | wc62 | model |
| Cascade Mask RCNN with FPN | R-50 | Pytorch | 0.485 | 0.365 | a8bl | model |
| SSD | VGG16 | Pytorch | 0.397 | - | uffe | model |
| Retinanet with FPN | R-50 | Pytorch | 0.368 | - | lfio | model |
| Retinanet with FPN | R-101 | Pytorch | 0.359 | - | p1rd | model |
| FoveaBox | R-101 | Pytorch | 0.389 | - | kwiq | model |
| FCOS with FPN | R-101 | Pytorch | 0.351 | - | 1djo | model |
Level 2
| Model | Backbone | Style | HBB mAP | SBB mAP | Extraction code | Download |
|---|---|---|---|---|---|---|
| Faster RCNN with FPN | R-50 | Pytorch | 0.345 | - | 924l | model |
| Faster RCNN with FPN | R-101 | Pytorch | 0.479 | - | fb1b | model |
| Mask RCNN with FPN | R-50 | Pytorch | 0.468 | 0.377 | so8j | model |
| Mask RCNN with FPN | R-101 | Pytorch | 0.488 | 0.398 | 7q1g | model |
| Cascade Mask RCNN with FPN | R-50 | Pytorch | 0.492 | 0.389 | t9gr | model |
| SSD | VGG16 | Pytorch | 0.423 | - | t1ma | model |
| Retinanet with FPN | R-50 | Pytorch | 0.369 | - | 4h0o | model |
| Retinanet with FPN | R-101 | Pytorch | 0.411 | - | g9ca | model |
| FoveaBox | R-101 | Pytorch | 0.427 | - | 8e12 | model |
| FCOS with FPN | R-101 | Pytorch | 0.431 | - | 0hl0 | model |
Level 3
| Model | Backbone | Style | HBB mAP | SBB mAP | Extraction code | Download |
|---|---|---|---|---|---|---|
| Faster RCNN with FPN | R-50 | Pytorch | 0.375 | - | 7qmo | model |
| Faster RCNN with FPN | R-101 | Pytorch | 0.543 | - | bmla | model |
| Mask RCNN with FPN | R-50 | Pytorch | 0.545 | 0.450 | a73h | model |
| Mask RCNN with FPN | R-101 | Pytorch | 0.564 | 0.472 | 7k9i | model |
| Cascade Mask RCNN with FPN | R-50 | Pytorch | 0.593 | 0.483 | ebga | model |
| SSD | VGG16 | Pytorch | 0.483 | - | otu5 | model |
| Retinanet with FPN | R-50 | Pytorch | 0.326 | - | tu5a | model |
| Retinanet with FPN | R-101 | Pytorch | 0.483 | - | ptv0 | model |
| FoveaBox | R-101 | Pytorch | 0.459 | - | 1acn | model |
| FCOS with FPN | R-101 | Pytorch | 0.498 | - | 40a8 | model |
Development kit
The ShipRSImageNet Development kit is based on DOTA Development kit and provides the following function
-
Load and image, and show the bounding box on it.
-
Covert VOC format label to COCO format label.
Citation
If you make use of the ShipRSImageNet dataset, please cite our following paper:
Z. Zhang, L. Zhang, Y. Wang, P. Feng and R. He, "ShipRSImageNet: A Large-Scale Fine-Grained Dataset for Ship Detection in High-Resolution Optical Remote Sensing Images," in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 14, pp. 8458-8472, 2021, doi: 10.1109/JSTARS.2021.3104230.
Contact
If you have any the problem or feedback in using ShipRSImageNet, please contact:
- Zhengning Zhang at 23880666@qq.com
License
ShipRSImageNet is released under the Apache 2.0 license. Please see the LICENSE file for more information.