Sentinel-1 polarimetric preprocessing

General information

Block type: processing (data preparation)
Supported input types:
- Sentinel1_l1c_grd (Sentinel 1 L1C GRD in SAFE format)
Provider: UP42
Tags: SAR, radar, C-Band, imagery, preprocessing, data preparation

Description

This repository contains the code implementing a block in UP42 that performs polarimetric processing of Synthetic Aperture Radar (SAR) with processing Level 1C and Ground Range Detection (GRD) — geo-referenced.

Inputs & outputs

This block takes as input a Level 1C GRD file and brings it into a format ready for analysis. It is based on ESA's SeNtinel Application Platform (SNAP). The applied processing steps are:

Value conversion: linear to dB.
Speckle filtering (using a median filter).
Creation of a land-sea mask.
Format conversion to GeoTIFF.
Apply terrain correction.

The output is a GeoTIFF file.

Block capabilities

The block takes a up42.data.scene.sentinel1_l1c_grd input capability and delivers up42.data.aoiclipped as output capability.

Requirements

docker.
GNU make.
Python: version >= 3.7 — only for local development.

Usage

Clone the repository

git clone https://github.com/up42/snap-polarimetric.git <directory>

where <directory> is the directory where the cloning is done.

Build the docker images

For building the images you should tag the image such that it can bu pushed to the UP42 docker registry, enabling you to run it as a custom block. For that you need to pass your user ID (UID) in the make command.

The quickest way to get that is just to go into the UP42 console and copy & paste from the last clipboard that you get at the custom-blocks page and after clicking on PUSH a BLOCK to THE PLATFORM. For example, it will be something like:

docker push registry.up42.com/<UID>/<image_name>:<tag>

Now you can launch the image building using make like this:

make build UID=<UID>

You can avoid selecting the exact UID by using pbpaste in a Mac (OS X) or xsel --clipboard --output in Linux and do:

# mac: OS X.
make build UID=$(pbpaste | cut -f 2 -d '/')

# Linux.
make build UID=$(xsel --clipboard --output | cut -f 2 -d '/')

You can additionaly specifiy a custom tag for your image (default tag is snap-polarimetric:latest):

make build UID=<UID> DOCKER_TAG=<docker tag>

if you don't specify the docker tag, it gets the default value of latest.

Push the image to the UP42 registry

You first need to login into the UP42 docker registry.

make login USER=me@example.com

where me@example.com should be replaced by your username, which is the email address you use in UP42.

Now you can finally push the image to the UP42 docker registry:

make push UID=<UID>

where <UID> is user ID referenced above. Again using the copy & pasting on the clipboard.

# mac: OS X.
make build UID=$(pbpaste | cut -f 2 -d '/')

# Linux.
make build UID=$(xsel --clipboard --output | cut -f 2 -d '/')

make push UID=<UID>

Note that if you specified a custom docker tag when you built the image, you need to pass it now to make.

make push UID=<UID> DOCKER_TAG=<docker tag>

where <UID> is user ID referenced above. Again using the copy & pasting on the clipboard.

# mac: OS X.
make build UID=$(pbpaste | cut -f 2 -d '/') DOCKER_TAG=<docker tag>

# Linux.
make build UID=$(xsel --clipboard --output | cut -f 2 -d '/') DOCKER_TAG=<docker tag>

After the image is pushed you should be able to see your custom block in the console and you can now use the block in a workflow.

Run the processing block locally

Configure the job

To run the docker image locally you need first to configure the job with the parameters specific to this block. Create a params.json like this:

{
  "polarisations": <array polarizations>,
  "mask": <array mask type>,
  "tcorrection": <boolean>
}

where:

<array polarizations>: JS array of possible polarizations: "VV", "VH", "HV", "HH".
<array of mask type>: JS array of possible mask "sea" or "land".
<boolean>: true or false stating if terrain correction is to be done or not.

Here is an example params.json:

{
  "polarisations": ["VV"],
  "mask": ["sea"],
  "tcorrection": false
}

Get the data

A radar image is needed for the block to run. Such image can be obtained by creating a workflow with a single Sentinel 1 L1C GRD data block and download the the result.

Then create the directory /tmp/e2e_snap_polarimetric/:

mkdir /tmp/e2e_snap_polarimetric

Now untar the tarball with the result in that directory:

tar -C /tmp/e2e_snap_polarimetric -zxvf <downloaded tarball>

Run the block

make run

If set a custom docker tag then the command ro run the block is:

make run DOCKER_TAG=<docker tag>

Local development

Install the required libraries

First create a virtual environment either by using virtualenvwrapper or virtualenv.

In the case of using virtualenvwrapper do:

mkvirtualenv -p $(which python3.7) up42-snap

In the case of using virtualenv do:

virtualenv -p $(which python3.7) up42-snap

After creating a virtual environment and activating it, all the necessary libraries can be installed on this environment by doing:

make install

Run the tests

This project uses pytest for testing. To run the tests, first create two empty /tmp/input/ and /tmp/output directories. The output will be written to the /tmp/output/ directory. Finally, to run the test do as following:

make test

Now you need to build and run the block locally.

Support

Open an issue here.
Reach out to us on gitter.
Mail us support@up42.com.

swordfish23/snap-polarimetric