wri/gfw-datapump

Write new AOIs into part files

thomas-maschler opened this issue · 0 comments

Is your feature request related to a problem? Please describe.
Geostore only supports bulk downloads for up to 100 features. We need to mitigate this

Describe the solution you'd like
In check new area lambda, split geostore ID list into chunks of 100 and download 100 features at a time only. Write results into PART files (ie user-aoi-yyyymmdd-partxxx.tsv)

We could consider to extract the download and export functionality from the current lambda function and move it to its own. This way, we could run multiple download/ export functions in parallel.