cultionet_field-_boundary_delineation: A Python repository from sumesh1

cultionet is a library for semantic segmentation of cultivated land using a neural network.

The library is built on PyTorch Geometric and PyTorch Lightning. The segmentation objectives (class targets and losses) were designed following the work by Waldner et al.. However, there are two key differences between this library and the paper above:

cultionet uses time series instead of individual dates for training and inference.
cultionet uses a mix of GNN and CNN layers.

The cultionet input data

The model inputs are satellite time series--they can be either bands or spectral indices. Data are stored in a PyTorch Geometric Data object. For example,

from torch_geometric.data import Data

batch = Data(x=x, y=y, edge_index=edge_index, edge_attrs=edge_attrs)

where

x = torch.Tensor of (samples x bands*time)
y = torch.Tensor of (samples,)

As an example, for a time series of red, green, blue, and NIR with 25 time steps (bi-weekly + 1 additional end point), the data would be shaped like:

x = [[r_w1, ..., r_w25, g_w1, ..., g_wN, b_w1, ..., b_wN, n_w1, ..., n_wN]]

Create train dataset

Create the training data

Training data should consist of two files per grid/year. One file is a polygon vector file (stored as a GeoPandas-compatible format like GeoPackage) of the training grid for a region. The other is a polygon vector file (stored in the same format) of the training labels for a grid.

What is a grid?

A grid defines an area to be labeled. For example, a grid could be 1 km x 1 km. A grid should be small enough to be combined with other grids in batches in GPU memory. Thus, 1 km x 1 km is a good size with, say, Sentinel-2 imagery at 10 m spatial resolution.

Note: grids across a study area should all be of equal dimensions

What is a training label?

Training labels are polygons of delineated crop (i.e., crop fields). The training labels will be clipped to the training grid (described above). Thus, it is important to exhaustively digitize all crop fields within a grid.

Configuration file

The configuration file (cultionet/scripts/config.yml) is used to create training datasets. This file is only meant to be a template. For each project, copy this template and modify it accordingly.

image_vis
- A list of image indices to use for training.
regions
- The start and end range of the training regions to use in the dataset.
years
- A list of years to use in the training dataset. Image years correspond to the end period of the time series. Thus, 2021 would align with a time 2020-2021 series.

Training data requirements

The polygon vector file should have a field named 'class', with values for crop fields set equal to 1. For grids with all null data (i.e., non-crop), simply create an empty polygon or a polygon matching the grid extent with 'class' value equal to 0.

Training name requirements

The polygon/grid pairs should be named {region}{poly}{year}.gpkg. The region name should be an integer of six character length (e.g., the region might correspond to grid 1 and be named '000001_poly_2020.gpkg'.

Example directory structure for training data.

project_dir:
  user_train:
    '{region}_grid_{time_series_end_year}.gpkg'
    '{region}_poly_{time_series_end_year}.gpkg'

Create the image time series

This must be done outside of cultionet. Essentially, a directory with band or VI time series must be generated before using cultionet.

Note: it is expected that the time series have length greater than 1

The raster files should be stored as GeoTiffs with names that follow the yyyyddd.tif format.
There is no maximum requirement on the temporal frequency (i.e., daily, weekly, bi-weekly, monthly, etc.).
- Just note that a higher frequency will result in larger memory footprints for the GPU plus slower training and inference.
While there is no requirement for the time series frequency, time series must have different start and end years.
- For example, a northern hemisphere time series might consist of (1 Jan 2020 to 1 Jan 2021) whereas a southern hemisphere time series might range from (1 July 2020 to 1 July 2021). In either case, note that something like (1 Jan 2020 to 1 Dec 2020) will not work.
The years in the directories must align with the training data files. More specifically, the training data year (year in the polygon/grid pairs) should correspond to the time series end year.
- For example, a file named 000001_poly_2020.gpkg should be trained on 2019-2020 imagery, while 000001_poly_2022.gpkg would match a 2021-2022 time series.
The image time series footprints (bounding box) can be of any size, but should encompass the training data bounds. During data creation (next step below), only the relevant bounds of the image are extracted and matched with the training data using the training grid bounds.

Example time series directory with bi-weekly cadence for three VIs (i.e., evi2, gcvi, kndvi)

project_dir:
   time_series_vars:
      region:
         evi2:
          2020001.tif
          2020014.tif
          ...
          2021001.tif
          2021014.tif
          ...
          2022001.tif
         gcvi:
          <repeat of above>
         kndvi:
          <repeat of above>

Create the time series training data

After training data and image time series have been created, the training data PyTorch files (.pt) can be generated using the commands below.

Note: Modify a copy of cultionet/scripts/config.yml as needed.

# Navigate to the cultionet script directory.
cd cultionet/scripts/
# Activate the virtual environment. See installation section below for environment details.
pyenv venv venv.cultionet
# Create the training dataset.
(venv.cultionet) cultionet create --project-path /project_dir --grid-size 100 100 --config-file config.yml

The output .pt data files will be stored in /project_dir/data/train/processed. Each .pt data file will consist of all the information needed to train the segmentation model.

Training a model

To train the model, you will need to create the train dataset object and pass it to cultionet fit method. A script is provided to help ease this process. To train a model on a dataset, use (as an example):

(venv.cultionet) cultionet train --project-path /project_dir --val-frac 0.2 --random-seed 500 --batch-size 4 --epochs 30 --filters 32 --device gpu --patience 5 --learning-rate 0.001 --reset-model

For more CLI options, see:

(venv.cultionet) cultionet train -h

Example usage of the cultionet API

In the examples below, we use the project path of the setup examples above to train a model using cultionet. Note that this is similar to using the CLI example above. The point here is simply to demonstrate the use of the Python API.

Fit a model using cultionet

import cultionet
from cultionet.data.datasets import EdgeDataset
from cultionet.utils.project_paths import setup_paths
from cultionet.utils.normalize import get_norm_values

# Fraction of data to use for model validation
# The remainder will be used for training
val_frac = 0.2
# The random seed|state used to split the data
random_seed = 42

# This is a helper function to manage paths
ppaths = setup_paths('project_dir')

# Get the normalization means and std. deviations
ds = EdgeDataset(ppaths.train_path)
cultionet.model.seed_everything(random_seed)
train_ds, val_ds = ds.split_train_val(val_frac=val_frac)
# Calculate the values needed to transform to z-scores, using
# the training data
data_values = get_norm_values(dataset=train_ds, batch_size=16)

# Create the train data object again, this time passing
# the means and standard deviation tensors
ds = EdgeDataset(
    ppaths.train_path,
    data_means=data_values.mean,
    data_stds=data_values.std
)

# Fit the model
cultionet.fit(
   dataset=ds,
   ckpt_file=ppaths.ckpt_file,
   val_frac=val_frac,
   epochs=30,
   learning_rate=0.001,
   filters=32,
   random_seed=random_seed
)

After a model has been fit, the last checkpoint file can be found at /project_dir/ckpt/last.ckpt.

Apply model inference to a single .pt data object

import cultionet
from cultionet.data.utils import create_network_data, create_network_dataset
from cultionet.utils.project_paths import setup_paths
from cultionet.utils import model_preprocessing
import numpy as np
from rasterio.windows import Window
import torch

# The random seed|state used to split the data
random_seed = 42

# This is a helper function to manage paths
ppaths = setup_paths('project_dir')

# Create the network data
nvars = 3   # number of variables (VIs) used to train the model
ntime = 25  # number of samples in 1 time series
nfeas = nvars * ntime   # total number of model features
height = 64     # number of image rows
width = 64      # number of image columns
# Create fake data as an example of the required dimensions
xvars = np.random.random((nfeas, height, width))
# Create the network data
data = create_network_data(xvars, ntime)
# Load the z-score norm values
data_values = torch.load(ppaths.norm_file)
# Create the temporary dataset
predict_ds = create_network_dataset(data, ppaths.predict_path, data_values)

# Apply inference
stack, lit_model = cultionet.predict(
                        predict_ds=predict_ds,
                        ckpt_file=ppaths.ckpt_file,
                        filters=32,
                        device='gpu',
                        w=Window(row_off=0, col_off=0, height=height, width=width),
                        w_pad=Window(row_off=0, col_off=0, height=height, width=width)
                    )

Installation

If using a GPU with CUDA 11.3, see the cultionet Dockerfile and dockerfiles/README.md to build a Docker image.

If installing from scratch locally, see the instructions below.

Requirements

Install NVIDIA driver

sudo add-apt-repository ppa:graphics-drivers/ppa
sudo apt-get update
sudo apt install ubuntu-drivers-common
ubuntu-drivers devices
sudo apt install nvidia-driver-465

reboot machine

Install CUDA toolkit

See https://developer.nvidia.com/cuda-11.3.0-download-archive?target_os=Linux&target_arch=x86_64&Distribution=Ubuntu&target_version=20.04&target_type=deb_local