GSA CoE Data Mesh Proof of Concept for USGS 3D Elevation Program (3DEP)

This project inherits from developmentseed/eoAPI and provides some customizations for the demo. See also their eoapi-cdk Infrastructure-as-Code implementation specific to the AWS platform.

Microservices provided

STAC

Dynamic Spatiotemporal Asset Catalog for geospatial metadata provided by pgSTAC (docs, github)

STAC API

REST API endpoint provided by the stac-fastapi Python library with the following STAC API extensions enabled:

Transaction supports the creation, editing, and deleting of STAC Items through POST, PUT, PATCH, and DELETE requests.
Collection Transaction does the same for Collections.
Filter provides an expressive mechanism for searching based on Item attributes.
Query adds a query parameter that allows additional filtering based on the properties of Item objects.
Sort allows the user to define the fields by which to sort results.
Fields describes a mechanism to include or exclude certain fields from a response.
Context gives more information about the number of items returned when performing a search versus the number of item that may match the search term but were not returned.

The API negotiates with the pgSTAC backend via the stac-fastapi-pgstac Python library.

STAC Browser

Vue.js front end to discover data products.
Demo should include limited front end changes to base UI to facilitate data mesh-style self-service data infrastructure workflow.

JupyterHub

Example notebooks provided to work with 3D elevation data as well as the other services provided.

Possble TBD features For Phase 2

Authentication

If assets are locked behind paywall/firewall, the Authentication STAC extension can be used (N.B. this is different from the STAC API extension below). This extension adds fields to STAC Items and STAC Catalogs to define authentication or authorization flows used to access Assets and Links behind security.
If you want to enable authentication within the STAC API to restrict the ability to write to the catalog to specific individuals/roles within the context of the API, the Development Seed team recommends securing applications by using JWTs as explained in this technical article

Original eoAPI README below:

Create a full Earth Observation API with Metadata, Raster, and Vector services.

Documentation: https://eoapi.dev

Source Code: https://github.com/developmentseed/eoAPI

Earth Observation API

eoAPI combines several state-of-the-art projects to create a full Earth Observation API. Each service can be used and deployed independently, but eoAPI creates the interconnections between each service:

pgSTAC database https://github.com/stac-utils/pgstac
STAC API built on top of https://github.com/stac-utils/stac-fastapi
STAC Items And Mosaic Raster Tiles API built on top of https://github.com/stac-utils/titiler-pgstac
OGC Features and Vector Tiles API built on top of https://github.com/developmentseed/tipg
A STAC Catalog browsing UI based on the radiant earth browser : https://github.com/radiantearth/stac-browser

🌍 eoAPI: An Open-Source Community Project

eoAPI is proudly open-source and driven by a dedicated community of contributors. We believe in the power of open collaboration and welcome anyone to contribute, discuss, and grow this tool. Join the conversations on GitHub Discussions and make a difference in the Earth Observation realm.

Getting started

The easiest way to start exploring the different eoAPI services is with Docker. Clone this repository and start the multi-container Docker applications using Compose:

git clone https://github.com/developmentseed/eoAPI.git
cd eoAPI
docker compose up

Once the applications are up, you'll need to add STAC Collections and Items to the PgSTAC database. If you don't have these available, you can follow the MAXAR open data demo (or get inspired by the other demos).

Then you can start exploring your dataset with:

the STAC Metadata service http://localhost:8081
the Raster service http://localhost:8082
the browser UI http://localhost:8085

If you've added a vector dataset to the public schema in the Postgres database, they will be available through the Vector service at http://localhost:8083.

Local deployment

The services can be deployed altogether locally with docker compose up.

Alternatively, you may install the libraries and launch the applications manually:

python -m pip install --upgrade virtualenv
virtualenv .venv
source .venv/bin/activate

export DATABASE_URL=postgresql://username:password@0.0.0.0:5439/postgis  # Connect to the database of your choice

python -m pip install uvicorn

###############################################################################
# Install and launch the application
# Select one of the following

###############################################################################
# STAC
python -m pip install "psycopg[binary,pool]" stac-fastapi-pgstac
python -m uvicorn stac_fastapi.pgstac.app:app --port 8081 --reload

###############################################################################
# RASTER
python -m pip install "psycopg[binary,pool]" titiler-pgstac
python -m uvicorn titiler.pgstac.main:app --port 8082 --reload

###############################################################################
# VECTOR
python -m pip install tipg
python -m uvicorn tipg.main:app --port 8083 --reload

Note: Python libraries might have incompatible dependencies, which you can resolve by using a virtual environment for each one.

Deployment on the cloud

Kubernetes

eoapi-k8s contains IaC and Helm charts for deploying eoAPI services on AWS and GCP.

AWS CDK

eoapi-cdk defines a set of AWS CDK constructs that can be used to deploy eoAPI services on AWS. An official example usage of these constructs can be found at eoapi-template.

Deployment with custom runtimes

An example of custom eoAPI runtimes and deployment can be found at eoapi-devseed.

Contribution & Development

We highly value and rely on our community! You can make a difference whether you're an expert or just getting started. Here's how:

Engage in Discussions: Share your ideas, ask questions, or provide feedback through GitHub Discussions. This is where most of our project conversations take place.
Report Issues: Found a bug or have a feature request? Raise it on our issues page.

License

At Development Seed, we believe in open collaboration and making tools and data more accessible. In line with this ethos, we've explicitly chosen a MIT license for eoAPI.

For full license details, see LICENSE.

Authors

Nurtured by Development Seed

See contributors for a listing of individual contributors.

colettace/eoAPI