openclimatefix/Satip

Speed up 15 mins

peterdudfield opened this issue · 2 comments

Detailed Description

I'm worried duplicated customization are being made, and that really slows the process of getting all the 15 mins data.

Currently at the start of the script we check to see if the files have already been made.
I wonder before each customization is made, could the app check to see if the file is in s3, if it is, load it, if not run the customization?

Context

  • when the satellite is on backup, it takes about 30 mins to get 1 hour of data.

Possible Implementation

See above

Yeah, that should be easy enough to change I think

Thanks, you ok to have a go at this?