/eemont

A python package that extends Google Earth Engine.

Primary LanguageJupyter NotebookMIT LicenseMIT

eemont

A Python package that extends Google Earth Engine

PyPI Version conda-forge Version License Documentation Status Tests Status Donation GitHub Sponsors Donation Buy me a coffee Donation kofi Downloads GEE Community Twitter JOSS Paper Black isort

Table of Contents

Overview

Google Earth Engine is a cloud-based service for geospatial processing of vector and raster data. The Earth Engine platform has a JavaScript and a Python API with different methods to process geospatial objects. Google Earth Engine also provides a HUGE PETABYTE-SCALE CATALOG of raster and vector data that users can process online (e.g. Landsat Missions Image Collections, Sentinel Missions Image Collections, MODIS Products Image Collections, World Database of Protected Areas, etc.). The eemont package extends the Google Earth Engine Python API with pre-processing and processing tools for the most used satellite platforms by adding utility methods for different Earth Engine Objects that are friendly with the Python method chaining.

Google Earth Engine Community: Developer Resources

The eemont Python package can be found in the Earth Engine Community: Developer Resources together with other awesome resources such as geemap and rgee.

Additional Resources

If you like eemont, you might be interested in...

  • Awesome Spectral Indices: A ready-to-use curated list of spectral indices for remote sensing applications.
  • spectral: Awesome Spectral Indices for the Google Earth Engine JavaScript API (Code Editor).
  • eeExtra: A ninja Python package behind rgee, rgeeExtra and eemont.
  • rgeeExtra: High-level functions to process spatial and simple Earth Engine objects.

How does it work?

The eemont python package extends the following Earth Engine classes:

New utility methods and constructors are added to above-mentioned classes in order to create a more fluid code by being friendly with the Python method chaining. These methods are mandatory for some pre-processing and processing tasks (e.g. clouds masking, shadows masking, image scaling, spectral indices computation, etc.), and they are presented as simple functions that give researchers, students and analysts the chance to analyze data with far fewer lines of code.

Look at this simple example where a Sentinel-2 Surface Reflectance Image Collection is pre-processed and processed in just one step:

import ee, eemont

ee.Authenticate()
ee.Initialize()

point = ee.Geometry.PointFromQuery('Cali, Colombia',user_agent = 'eemont-example') # Extended constructor

S2 = (ee.ImageCollection('COPERNICUS/S2_SR')
    .filterBounds(point)
    .closest('2020-10-15') # Extended (pre-processing)
    .maskClouds(prob = 70) # Extended (pre-processing)
    .scaleAndOffset() # Extended (pre-processing)
    .spectralIndices(['NDVI','NDWI','BAIS2'])) # Extended (processing)

And just like that, the collection was pre-processed, processed and ready to be analyzed!

Installation

Install the latest eemont version from PyPI by running:

pip install eemont

Upgrade eemont by running:

pip install -U eemont

Install the development version from GitHub by running:

pip install git+https://github.com/davemlz/eemont

Install the latest eemont version from conda-forge by running:

conda install -c conda-forge eemont

Features

Let's see some of the main features of eemont and how simple they are compared to the GEE Python API original methods:

Overloaded Operators

The following operators are overloaded: +, -, *, /, //, %, **, <<, >>, &, |, <, <=, ==, !=, >, >=, -, ~. (and you can avoid the ee.Image.expression() method!)

GEE Python API eemont-style 
ds = 'COPERNICUS/S2_SR'

S2 = (ee.ImageCollection(ds)
  .first())

def scaleImage(img):
    scaling = img.select('B.*')
    x = scaling.multiply(0.0001)
    scaling = img.select(['AOT','WVP'])
    scaling = scaling.multiply(0.001)
    x = x.addBands(scaling)
    notScaling = img.select([
        'SCL',
        'TCI.*',
        'MSK.*',
        'QA.*'
    ]))
    return x.addBands(notScaling)

S2 = scaleImage(S2)

exp = '2.5*(N-R)/(N+(6*R)-(7.5*B)+1)'

imgDict = {
  'N': S2.select('B8'),
  'R': S2.select('B4'),
  'B': S2.select('B2')
}

EVI = S2.expression(exp,imgDict)
 
ds = 'COPERNICUS/S2_SR'

S2 = (ee.ImageCollection(ds)
  .first()
  .scale())

N = S2.select('B8')
R = S2.select('B4')
B = S2.select('B2')

EVI = 2.5*(N-R)/(N+(6*R)-(7.5*B)+1)

Clouds and Shadows Masking

Masking clouds and shadows can be done using eemont with just one method: maskClouds()!

GEE Python API eemont-style 
ds = 'LANDSAT/LC08/C01/T1_SR'

def maskCloudsShadows(img):
    c = (1 << 3)
    s = (1 << 5)
    qa = 'pixel_qa'
    qa = img.select(qa)
    cm = qa.bitwiseAnd(c).eq(0)
    sm = qa.bitwiseAnd(s).eq(0)
    mask = cm.And(sm)
    return img.updateMask(mask)

(ee.ImageCollection(ds)
  .map(maskCloudsShadows))
 
ds = 'LANDSAT/LC08/C01/T1_SR'

(ee.ImageCollection(ds)
  .maskClouds())

Image Scaling and Offsetting

Scaling and offsetting can also be done using eemont with just one method: scale()!

GEE Python API eemont-style 
def scaleBands(img):
    scaling = img.select([
      'NDVI',
      'EVI',
      'sur.*'
    ])
    x = scaling.multiply(0.0001)
    scaling = img.select('.*th')
    scaling = scaling.multiply(0.01)
    x = x.addBands(scaling)
    notScaling = img.select([
      'DetailedQA',
      'DayOfYear',
      'SummaryQA'
    ])
    return x.addBands(notScaling)

ds = 'MODIS/006/MOD13Q1'

(ee.ImageCollection(ds)
  .map(scaleBands))
 
ds = 'MODIS/006/MOD13Q1'

(ee.ImageCollection(ds)
  .scale())

Complete Preprocessing

The complete preprocessing workflow (Masking clouds and shadows, and image scaling and offsetting) can be done using eemont with just one method: preprocess()!

GEE Python API eemont-style 
ds = 'LANDSAT/LC08/C01/T1_SR'

def maskCloudsShadows(img):
    c = (1 << 3)
    s = (1 << 5)
    qa = 'pixel_qa'
    qa = img.select(qa)
    cm = qa.bitwiseAnd(c).eq(0)
    sm = qa.bitwiseAnd(s).eq(0)
    mask = cm.And(sm)
    return img.updateMask(mask)

def scaleBands(img):
    scaling = img.select('B[1-7]')
    x = scaling.multiply(0.0001)
    scaling = img.select([
      'B10','B11'
    ])
    scaling = scaling.multiply(0.1)
    x = x.addBands(scaling)
    notScaling = img.select([
      'sr_aerosol',
      'pixel_qa',
      'radsat_qa'
    ])
    return x.addBands(notScaling)

(ee.ImageCollection(ds)
  .map(maskCloudsShadows)
  .map(scaleBands))
 
ds = 'LANDSAT/LC08/C01/T1_SR'

(ee.ImageCollection(ds)
  .preprocess())

Spectral Indices

Do you need to compute several spectral indices? Use the index() method! A lot of built-in vegetation, burn, water, snow, drought and kernel indices can be computed:

GEE Python API eemont-style 
ds = 'LANDSAT/LC08/C01/T1_SR'

def scaleImage(img):
    scaling = img.select('B[1-7]')
    x = scaling.multiply(0.0001)
    scaling = img.select(['B10','B11'])
    scaling = scaling.multiply(0.1)
    x = x.addBands(scaling)
    notScaling = img.select([
        'sr_aerosol',
        'pixel_qa',
        'radsat_qa'
    ]))
    return x.addBands(notScaling)

def addIndices(img):
    x = ['B5','B4']
    a = img.normalizedDifference(x)
    a = a.rename('NDVI')
    x = ['B5','B3']
    b = img.normalizedDifference(x)
    b = b.rename('GNDVI')
    x = ['B3','B6']
    c = img.normalizedDifference(x)
    c = b.rename('NDSI')
    return img.addBands([a,b,c])

(ee.ImageCollection(ds)
  .map(scaleImage)
  .map(addIndices))
 
ds = 'LANDSAT/LC08/C01/T1_SR'

(ee.ImageCollection(ds)
  .scale()
  .index(['NDVI','GNDVI','NDSI']))

The list of available indices can be retrieved by running:

eemont.listIndices()

Information about the indices can also be checked:

indices = eemont.indices()
indices.BAIS2.formula
indices.BAIS2.reference

Closest Image to a Specific Date

Struggling to get the closest image to a specific date? Here is the solution: the closest() method!

GEE Python API eemont-style 
ds = 'COPERNICUS/S5P/OFFL/L3_NO2'

xy = [-76.21, 3.45]
poi = ee.Geometry.Point(xy)

date = ee.Date('2020-10-15')
date = date.millis()

def setTimeDelta(img):
    prop = 'system:time_start'
    prop = img.get(prop)
    prop = ee.Number(prop)
    delta = prop.subtract(date)
    delta = delta.abs()
    return img.set(
      'dateDist',
      delta)

(ee.ImageCollection(ds)
  .filterBounds(poi)
  .map(setTimeDelta)
  .sort('dateDist')
  .first())
 
ds = 'COPERNICUS/S5P/OFFL/L3_NO2'

xy = [-76.21, 3.45]
poi = ee.Geometry.Point(xy)

(ee.ImageCollection(ds)
  .filterBounds(poi)
  .closest('2020-10-15'))

Time Series By Regions

The JavaScript API has a method for time series extraction (included in the ui.Chart module), but this method is missing in the Python API... so, here it is!

PD: Actually, there are two methods that you can use: getTimeSeriesByRegion() and getTimeSeriesByRegions()!

f1 = ee.Feature(ee.Geometry.Point([3.984770,48.767221]).buffer(50),{'ID':'A'})
f2 = ee.Feature(ee.Geometry.Point([4.101367,48.748076]).buffer(50),{'ID':'B'})
fc = ee.FeatureCollection([f1,f2])

S2 = (ee.ImageCollection('COPERNICUS/S2_SR')
   .filterBounds(fc)
   .filterDate('2020-01-01','2021-01-01')
   .maskClouds()
   .scale()
   .index(['EVI','NDVI']))

# By Region
ts = S2.getTimeSeriesByRegion(reducer = [ee.Reducer.mean(),ee.Reducer.median()],
                              geometry = fc,
                              bands = ['EVI','NDVI'],
                              scale = 10)

# By Regions
ts = S2.getTimeSeriesByRegions(reducer = [ee.Reducer.mean(),ee.Reducer.median()],
                               collection = fc,
                               bands = ['EVI','NDVI'],
                               scale = 10)

Constructors by Queries

Don't you have the coordinates of a place? You can construct them by using queries!

usr = 'my-eemont-query-example'

seattle_bbox = ee.Geometry.BBoxFromQuery('Seattle',user_agent = usr)
cali_coords = ee.Feature.PointFromQuery('Cali, Colombia',user_agent = usr)
amazonas_river = ee.FeatureCollection.MultiPointFromQuery('Río Amazonas',user_agent = usr)

Supported Platforms

The Supported Platforms for each method can be found in the eemont documentation.

  • Masking clouds and shadows supports Sentinel Missions (Sentinel-2 SR and Sentinel-3), Landsat Missions (SR products) and some MODIS Products. Check all details in User Guide > Masking Clouds and Shadows > Supported Platforms.
  • Image scaling supports Sentinel Missions (Sentinel-2 and Sentinel-3), Landsat Missions and most MODIS Products. Check all details in User Guide > Image Scaling > Supported Platforms.
  • Spectral indices computation supports Sentinel-2 and Landsat Missions. Check all details in User Guide > Spectral Indices > Supported Platforms.
  • Getting the closest image to a specific date and time series supports all image collections with the system:time_start property.

License

The project is licensed under the MIT license.

Contributing

Contributions to eemont are welcome! Here you will find how to do it:

  • Bugs: If you find a bug, please report it by opening an issue. if possible, please attach the error, code, version, and other details.
  • Fixing Issues: If you want to contributte by fixing an issue, please check the eemont issues: contributions are welcome for open issues with labels bug and help wanted.
  • Enhancement: New features and modules are welcome! You can check the eemont issues: contributions are welcome for open issues with labels enhancement and help wanted.
  • Documentation: You can add examples, notes and references to the eemont documentation by using the NumPy Docstrings of the eemont documentation, or by creating blogs, tutorials or papers.

Contribution Steps

First, fork the eemont repository and clone it to your local machine. Then, create a development branch:

git checkout -b name-of-dev-branch

eemont is divided according to Earth Engine classes, and you will find a module for each class (e.g. imagecollection.py). Look for the required class as follows:

  • ee.Feature: feature.py
  • ee.FeatureCollection: featurecollection.py
  • ee.Geometry: geometry.py
  • ee.Image: image.py
  • ee.ImageCollection: imagecollection.py

The common.py is used for methods that can be used for more than one Earth Engine class.

When creating new features, please start with the self argument and add the corresponding decorator ( @extend() from the extending module). Check this example:

from .extending import extend

@extend(ee.image.Image, static = False)
def my_new_method(self,other):
     '''Returns the addition of and image and a float.

     Parameters
     ----------
     self : ee.Image [this]
         Image to add.
     other : float
         Float to add.

     Returns
     -------
     ee.Image
         Addition of an ee.Image and a float.

     Examples
     --------
     >>> import ee, eemont
     >>> ee.Initialize()
     >>> img = ee.Image(0).my_new_method(other = 3.14)
     '''
     return self.add(other)

By using the @extend() decorator, the my_new_method() method is added to the ee.Image class. If you want to add a static method, please set the static argument to False. Look for the required class as follows:

  • ee.Feature: ee.feature.Feature
  • ee.FeatureCollection: ee.featurecollection.FeatureCollection
  • ee.Geometry: ee.geometry.Geometry
  • ee.Image: ee.image.Image
  • ee.ImageCollection: ee.imagecollection.ImageCollection
  • ee.List: ee.ee_list.List
  • ee.Number: ee.ee_number.Number

Remember to use Black!

In order to test additions, you can use pytest over the tests folder:

pytest tests

This will autmatically test all modules for the available satellite platforms through eemont. If you have added a new feature, please include it in the tests.

To test across different Python versions, please use tox.

Now it's time to commit your changes and push your development branch:

git add .
git commit -m "Description of your work"
git push origin name-of-dev-branch

And finally, submit a pull request.

How to cite

Do you like using eemont and think it is useful? Share the love by citing it!:

Montero, D., (2021). eemont: A Python package that extends Google Earth Engine. Journal of Open Source Software, 6(62), 3168, https://doi.org/10.21105/joss.03168

If required, here is the BibTex!:

@article{Montero2021,
  doi = {10.21105/joss.03168},
  url = {https://doi.org/10.21105/joss.03168},
  year = {2021},
  publisher = {The Open Journal},
  volume = {6},
  number = {62},
  pages = {3168},
  author = {David Montero},
  title = {eemont: A Python package that extends Google Earth Engine},
  journal = {Journal of Open Source Software}
}

Artists

  • David Montero Loaiza: Lead Developer of eemont and eeExtra.
  • César Aybar: Lead Developer of rgee and eeExtra.
  • Aaron Zuspan: Plus Codes Constructors and Methods, Panchromatic Sharpening and Histogram Matching Developer.

Credits

Special thanks to Justin Braaten for featuring eemont in tutorials and the GEE Community: Developer Resources Page, to César Aybar for the formidable help with Awesome Spectral Indices for GEE and to the JOSS Review Team (Katy Barnhart, Jayaram Hariharan, Qiusheng Wu and Patrick Gray) for the comments, suggestions and contributions!