knowledge-computing/sigspatial-cup-2023

The code and result for 12th SIGSPATIAL Cup (GISCUP 2023)

ACM SIGSPATIAL Cup 2023

The code and result for 12th SIGSPATIAL Cup (GISCUP 2023) https://sigspatial2023.sigspatial.org/giscup/index.html

The goal is to detect supraglacial lakes on the Greenland ice sheet from satellite imagery. Our team proposes an ensembled approach that leverages two machine learning models, computer vision techniques, and external data (e.g., topographic sinks, soil) to automatically detect surface lakes.

Link to the submission GPKG: https://github.com/knowledge-computing/sigspatial-cup-2023/blob/main/GPKG/lake_polygons_test.gpkg

The content is organized as follows:

1. Data Preprocessing
2. Image Segmentation Models
   • 2.1. Segment Anything Model
   • 2.2. DeepLabv3+
3. External Data Resources
   • 3.1. Topographic Sink
   • 3.2. Soil Data
4. Data Postprocessing

---

1. Data Preprocessing

Directory ./data_preprocess/

The data preprocssing creates a training dataset from the provided competition data for machine learnning models. The generated datasets include region images (region_images), training set (train_crop1024_shift512), inference set (target_crop1024_shift512). You can download these datasets and organize them as THIS FOLDER (unzipping is required). You can then avoid the following steps and directly go to Image Segmentation Models.

How to run

• To produce the images for each region (i.e., crop the original tif into 6 regions), run python extract_regions.py --data_root [DATA_ROOT] --output_path [OUTPUT_PATH] --tif_file [TIF_FILE]

  • DATA_ROOT: data directory of the competition data
  • OUTPUT_DIR: output directory, default: ../dataset/region_images
  • TIF_FILE: the TIF file name you want to process, e.g., Greenland26X_22W_Sentinel2_2019-08-25_29.tif

• To crop the region images to image patches, run python crop_regions.py --data_root [DATA_ROOT] --crop_size [CROP_SIZE] --shift_size [SHIFT_SIZE]

  • DATA_ROOT: data root of the region images, e.g., ../dataset/
  • CROP_SIZE: cropped image size, default 1024
  • SHIFT_SIZE: shift size, default 512

• To convert the ground truth polygons in the gpkg file to image patch segmentation masks, run python construct_train_annotation.py --data_root [DATA_ROOT] --output_path [OUTPUT_PATH] --crop_size [CROP_SIZE] --shift_size [SHIFT_SIZE]

  • DATA_ROOT: data directory of the competition data
  • OUTPUT_PATH: output path, default: e.g., ../dataset/train_crop1024_shift512
  • CROP_SIZE: cropped image size, default 1024
  • SHIFT_SIZE: shift size, default 1024

• To generate training samples, run `python generate_train_set.py --data_root [DATA_ROOT]

  • DATA_PATH: data directory path, e.g., ../dataset/train_crop1024_shift512
  • You need hard_neg_images.json, hard_neg_images.txt, and invalid_image.txt under DATA_PATH

• To generate K-fold training and testing samples, run `python generate_train_test_set.py --data_root [DATA_ROOT]

  • DATA_PATH: data directory path, e.g., ../dataset/train_crop1024_shift512
  • You need hard_neg_images.json, hard_neg_images.txt, and invalid_image.txt under DATA_PATH
    

---

2. Image Segmentation Models

Description

We formulate the problem as a pixel-level binary-class classification problem, where supraglacial lakes should be predicted as 1 and backgrounds as 0. We use the provided lake labels to construct a training set of positive samples (containing lakes), negative samples ((no lake), and hard negative samples (no lake but having high variability in image color or some detection failures). Due to imbalanced positive and negative samples, we use the weighted random sampler to ensure balanced positive and negative samples in each training batch. We fine-tuned two machine learning models for the task: Facebook's Segment Anything Model (SAM) and DeepLabv3+.

2.1. Segment Anything Model

Directory ./models/SAM/

Model Weights You can download the model weights from HERE.

Environment Setup

• The model is trained with python 3.11 and CUDA 11.3
• To install the environment, pip install -r environment.yml

How to run

• To train, run train.py --epoch [EPOCH] --batch [BATCH] --lr [LR] --img_dir [IMG_DIR] --mask_dir [MASK_DIR] --postive_file [POSITIVE_FILE] --hard_negative_file [HARD_NEGATIVE_FILE]
  • EPOCH: the maximum number of training epoches
  • BATCH: batch size
  • IMG_DIR: the directory of training images
  • MASK_DIR: the directory of corresponding masks
  • POSITIVE_FILE: the txt file containing positive samples
  • HARD_NEGATIVE_FILE: the txt file containing hard negative samples
• To test/inference, run test.py --region [REGION] --model [MODEL] --img_dir [IMG_DIR] --test_file [TEST_FILE]
  • REGION: the region name to generate segmentation masks
  • MODEL: the model weight, i.e., model checkpoint (.pth file)
  • IMG_DIR: the directory of testing images
  • TEST_FILE: the txt file containing test images

2.2. DeepLabv3+

Directory ./models/DeepLabv3Plus/

Model Weights You can download the model weights from HERE. Please place the downloaded model weights under ./models/DeepLabv3Plus/weight/

Environment Setup

• The model is trained with python 3.8 and CUDA 11.3
• To install the environment in conda, run conda install pytorch==1.10.0 torchvision==0.11.0 torchaudio==0.10.0 cudatoolkit=11.3 -c pytorch -c conda-forge, then pip install -r requirements.txt
  

How to run

• To modify configuration file ./configs/config.yml
  
  • base_path: the root directory of image and mask folder
    
  • region_txt_base_path: the directory of txt files indicating positive and negative samples, named as train_pos.txt, train_neg.txt, test.txt (if needs)
    
  • save_res_path: the directory to the segmentation results
    
    
• To train, run python main.py -c configs/config.yml --train
  
• To test, run python main.py -c configs/config.yml --predict_on_test
  
• To inference given images, run python main.py -c configs/config.yml --predict --filefolder [IMAGES_DIR_PATH]
  

---

3. External Data Resources

3.1. Topographic Sink

Since lakes are defined by open water of some depth, we use the ArcticDEM to identify the potential locations for lakes. We generate topographic sinks to postprocess the model output. You can download the file HERE.

The process of generating topographic sinks from ArcticDEM has two steps. First, we employ the open-source WhiteboxToolsTM library to fill the depressions in the ArcticDEM and eliminate flat areas. Second, we generate topographic sinks by subtracting the output of the first step from the original ArcticDEM. Locations, where the subtraction results yield values smaller than zero, represent the topographic sinks.

3.2. Soil Data

We use the soil information to further illiminate the detected lakes that are not located in the glacier area. We use Northern Circumpolar Soil Carbon Database version 2 (NCSCDv2), a geospatial database that records the amount of organic carbon storage in soils of the northern circumpolar permafrost region down to a depth of 300 cm. Since the dataset delimited the areas that are covered by glaciers for most times in a year, we use this dataset to identify the glacier area and exclude lakes that are not located on glaciers. You can download the file HERE. Besides, we find that there are mountain regions that soil data do not cover, so we manually labeled and created a mountain mask as the external human knowledge to illiminate the wrongly detected lakes. The file is at ./data_postprocess/external_data/mountain_mask.csv.

---

4. Data Postprocessing

Directory ./data_postprocess/

After generating the segmentation results from the two models (you can find these intermediate results HERE), our approach first merges and extracts polygons from the segmentation masks respectively, and does some priliminary preprocesses on the polygons. Second, our approach treats the union of topographic sink, color thresholding, and the inverse of soil allocation as lake candidates, and removes all the model-based polygons that are not in the lake candidate. Then our approach compares the SAM-based results and topographic sink on a vector-wise basis. For each lake candidate, our approach only keeps the SAM-based polygon that has the largest overlapping area with the color-thresholding polygon. Lastly, our approach adds model-based polygons (from SAM and DeepLab that were removed) that reach the relative-area criteria of the lake candidates.

The preprocessed external datasets can be downloaded from HERE. You need to put it under ./data_postprocess/.

How to run

• To merge the segmentation results of image patches, run python postprocessing_merge.py --data_root [DATA_ROOT] --result_path [RESULT_PATH] --crop_size [CROP_SIZE] --shift_size [SHIFT_SIZE] --out_gpkg_path [OUT_GPKG_PATH]

  • DATA_ROOT: the data directory of the competition data
  • RESULT_PATH: the segmentation result path, e.g., ../results/deeplabv3p_update/2019-08-25_29_r5
  • CROP_SIZE: cropped image size, default 1024
  • SHIFT_SIZE: shift size, default 1024
  • OUT_GPKG_PATH: the directory of output gpkg files for each region

• To generate a single GPKG file that integrates results from different models, using topo-based and color-based extraction as a reference to postprocess, run python postprocessing_with_external.py --data_root [DATA_ROOT] --result_name [RESULT_NAME] --data_topo [DATA_TOPO] --data_soil [DATA_SOIL] --sam_dir [SAM_DIR] --dpl_dir [DPL_DIR]

  • DATA_ROOT: the data directory of the competition data
  • RESULT_NAME: the name of the output GPKG file
  • DATA_TOPO: the path to topographic_sink.tif
  • DATA_SOIL: the path to NCSCDv2_Greenland_WGS84_nonsoil_pct_0012deg.tif
  • SAM_DIR: the directory path of the results from the SAM model
  • DPL_DIR: the directory path of the results from the DeepLab model