Consider modifying lambda to trigger after metadata submission
Opened this issue · 2 comments
@milen-sage this was something that came out of the 2/19 IDP meeting -- would you be able to think it over and let us know if this something we should include in the first data release process?
We are considering what's the best way to trigger the image pre-processing script and where the latter would be hosted, since the pre-processing computation will not be executed in the lambda function itself (the lambda function is too resource-constrained). There are different options including AWS Batch and @clarisse-lau is investigating them.
Depending on the option she picks, we can decide on what set of conditions would actually trigger the pre-processing computation (e.g. story file and image file are submitted; and/or metadata manifest file is submitted). For now we are just considering triggering computation when both required inputs to the pre-processing script are provided: image + story file.