Ross-Alexandra/BYOA-Cloud-Compute

Partition input data

Closed this issue · 1 comments

The data needs to be divided into separate catalogs that can still be traversed/ STAC compliant. Currently the plan is to use a static amount of 2-4 partitions depending on how hard it is to split an active or static catalog and reconstruct it.

Main logic can be done anytime but full implementation will require data to be properly fetched from a datastore (see G-35).
This might be separated into a logic and implementation ticket depending on project status.

Acceptance Criteria

  • 2-4 even(ish) partions are created
  • each partition is a crawlable static/active STAC catalog
  • All partitions should be reference only. ie no data is cloned

Completed as part of the final push.