Partition input data
Closed this issue · 1 comments
smithaustin commented
The data needs to be divided into separate catalogs that can still be traversed/ STAC compliant. Currently the plan is to use a static amount of 2-4 partitions depending on how hard it is to split an active or static catalog and reconstruct it.
Main logic can be done anytime but full implementation will require data to be properly fetched from a datastore (see G-35).
This might be separated into a logic and implementation ticket depending on project status.
Acceptance Criteria
- 2-4 even(ish) partions are created
- each partition is a crawlable static/active STAC catalog
- All partitions should be reference only. ie no data is cloned
Ross-Alexandra commented
Completed as part of the final push.