Add update mechanism
jonblack opened this issue · 5 comments
The update scripts produce models before they are needed so future requests are quick. When a model doesn't exist, it's produced on-the-fly which is time consuming.
The update script was being run on bamboo but the produced files were never used by the frontend on chelonium. The service has since been removed from bamboo because it makes more sense to run the script on chelonium.
The docker scripts need updating to add the updating as a separate process. Care needs to be taken to ensure it doesn't slow the machine down too much.
I removed the batch update script as it's not the best approach to the problem. We also need to concentrate on a stable, working service before updating models in the background.
Once the service is stable we can address this issue and add an update mechanism.
The solution in the master branch for whynot, which uses APScheduler, is a good option. @cbaakman Is this feasible? How would hommod-rest know what to update?
APScheduler looks fine. Hommod rest knows what to update by looking at the mtime of the model archive files.
This should be part of the databanks scripts. These scripts should periodically create some models (not all because that would consume too many resources. Just a subset every now and then is fine).