Table of Contents generated with DocToc
- Sentinel1-SLC Product description
- Dataset in S3 at glance
- Querying for available imagery in the bucket
- Acknowledgements
Synthetic Aperture Radar (SAR) is an active remote sensing technique, where electromagnetic pulses are emitted and received by an antenna. The Sentinel-1 satellite mission consists of two identical satellites, Sentinel-1A and Sentinel-1B, launched by the European Space Agency (ESA) in 03 April 2014 and 25 April 2016 respectively (ESA, 2021a). These satellites orbit the Earth while acquiring SAR images at a wave frequency of 5.405GHz (C-band). This mission exploits the usage of 2 satellites in order to have a fast revisit time (12 days and down to 6 days in some areas).The main applications of the Sentinel-1 SAR images are the monitoring of land use changes and surface deformation along with support for emergency management. Further applications include but are not limited to monitoring of sea ice, icebergs, land ice, inland waters, oil spills, ships, and others (ESA, 2021a).
ESA. (2021a). About copernicus sentinel-1
The Sentinel-1 SLC IW image collection provided by ESA comes in an archive format, which must be entirely downloaded and unzipped first, in order to be used. Often times users only need selective data inside these archived folders for their work.
- The Earth On AWS dataset is stored in the unzipped form in the S3 bucket, offering users the option to selectively retrieve either the full imagery or only the parts of the data that is needed for a given study area.
- Since the dataset resides on S3, depending upon the application, users can also directly read the object into memory and carry out their work without having to download, unzip and store them in on-premise or cloud storages.
- The S3 bucket and objects in it are public. Anonymous access is also enabled. So users can access the data without aws account/credentials as well.
We have fully ingested Sentinel-1A/B Level-1 SLC over Germany, which is updated in the interval of 6 days, after they are made available by Alaska Satellite Facility (ASF).
Following up, as a next step, we aim to expand the service by increasing data coverage across the Europe region. The map below clearly describes the data coverage expansion plan.
The following figure provides an insight of the current percentage of EU dataset uploaded to the S3 bucket. As mentioned above, in addition to Germany, other regions are also being updated simultaneously, however, the process is still in progress with 34.31% of the total EU dataset already available in sentinel1-slc S3 bucket.
| Year | Percentage of EU dataset in S3 |
|---|---|
| 2014 | 94.55% |
| 2015 | 66.48% |
| 2016 | 24.08% |
| 2017 | 28.33% |
| 2018 | 29.53% |
| 2019 | 68.47% |
| 2020 | 19.80% |
| 2021 | 17.30% |
| 2022 | 43.21% |
| Total | 34.31% |
The dataset in the S3 bucket is organized in a directory structure based on the start date of the acquisition for ease of retrieval. The table below provides information on product details, useful for querying the dataset.
s3://sentinel1-slc/YYYY/MM/DD/XXX_BB_SLC__1SPP_YYYYMMDDTHHMMSS_yyyymmddThhmmss_OOOOOO_DDDDDD_CCCC.SAFE
Example S3 URI of an imagery:
s3://sentinel1-slc/2022/01/01/S1A_IW_SLC__1SDV_20220101T053305_20220101T053332_041263_04E777_0EE8.SAFE/
Each Sentinel-1 SAR product folder (object in the bucket) includes:
- a 'manifest.safe' file which holds the general product information in XML
- a measurement folder with complex measurement data set in GeoTIFF format per sub-swath per polarisation
- a preview folder containing 'quicklooks' in PNG format, Google Earth overlays in KML format and HTML preview files
- an annotation folder containing the product metadata in XML as well as calibration data
- a support folder containing the XML schemes describing the product XML.
| Variable | Description | Details (code: code details) |
|---|---|---|
XXX |
Denotes the satellite | S1A: Sentinel-1A S1B: Sentinel-1B |
BB |
Acquisition Mode | IW: Interferometric Wide-Swath |
PP |
Polarisation | SH:single HH polarisation SV: single VV polarisation DH: dual HH+HV polarisation DV: dual VV+VH polarisation HH: Partial Dual polarisation, HH only HV: Partial Dual polarisation, HV only VV: Partial Dual polarisation, VV only VH: Partial Dual polarisation, VV only |
YYYYMMDD |
Acquisition Start Date (UTC) | Four-digit year, two-digit month, two-digit day |
HHMMSS |
Acquisition Start Time (UTC) | Two-digit hour, two-digit minutes, two-digit seconds |
yyyymmdd |
Acquisition End Date (UTC) | Four-digit year, two-digit month, two-digit day |
hhmmss |
Acquisition End Time (UTC) | Two-digit hour, two-digit minutes, two-digit seconds |
OOOOOO |
Absolute orbit number at product start time | In the range of 000001-999999 |
DDDDDD |
Mission data take ID | In the range 000001-FFFFFF |
CCCC |
Hexadecimal string generated from CRC-16 of the manifest file | CRC-16 algorithm used to compute the unique identifier is CRC-CCITT (0xFFFF) |
The data in the S3 bucket can be queried with AWS CLI or with boto3 library.
For access without using the aws credentials, simply pass the --no-sign-request to the same command.
- To retrieve list of all years for which the imagery are available:
aws s3 ls sentinel1-slc aws s3 ls sentinel-slc --no-sign-request
- To retrieve list of all imagery available for a given year, month and date:
aws s3 ls sentinel1-slc/2022/01/01/ aws s3 ls sentinel1-slc/2022/01/01/ --no-sign-request
- To retrieve list of all files and folders inside a given imagery:
aws s3 ls sentinel1-slc/2022/01/01/S1A_IW_SLC__1SDV_20220101T053305_20220101T053332_041263_04E777_0EE8.SAFE/ aws s3 ls sentinel1-slc/2022/01/01/S1A_IW_SLC__1SDV_20220101T053305_20220101T053332_041263_04E777_0EE8.SAFE/ --no-sign-request
- To retrieve list of all imagery with a given polarization type and mission type:
aws s3 ls sentinel1-slc/2022/01/01/S1A_IW_SLC__1SDV_ aws s3 ls sentinel1-slc/2022/01/01/S1A_IW_SLC__1SDV_ --no-sign-request
-
Anonymously read objects without downloading and without passing AWS credentials.
import boto3 from botocore import UNSIGNED from botocore.config import Config s3_client = boto3.client('s3', config=Config(signature_version=UNSIGNED)) my_bucket = 'sentinel1-slc' file_to_read = '2022/01/01/S1A_IW_SLC__1SDV_20220101T053305_20220101T053332_041263_04E777_0EE8.SAFE/annotation/s1a-iw1-slc-vh-20220101t053305-20220101t053330-041263-04e777-001.xml' s3_response_object = s3_client.get_object(Bucket=my_bucket, Key=file_to_read) object_content = s3_response_object['Body'].read() print(object_content)
-
Retrieve list of all imagery for a given year and month with AWS credentials.
import boto3 client = boto3.client('s3', region_name='eu-west-1') my_bucket = 'sentinel1-slc' prefix_to_query = '2022/01/01/' results = client.list_objects(Bucket=my_bucket, Prefix=prefix_to_query, Delimiter='/' ) for result in results.get('CommonPrefixes'): print(result.get('Prefix'))
Tools and ready to use scripts will be added to the repository in future updates.
We would like to thank Amazon Web Services for providing the storage resources for this program. We would also like to acknowledge the team at ASF for their collaboration.