bloer/bgexplorer

prevent multiple processes performing cachable calculations

bloer opened this issue · 0 comments

bloer commented

When a new model is saved, we launch a background thread to cache a bunch of calculation outputs, namely the datatable and spectra. If the user then tries to view the model while the cache is being generated, we will have multiple processes all trying to do the same calculations.

For spectra, this isn't too bad, because each process can benefit from cache entries made by the other processes. But the datatable is only cached at the end of the calculation, so it could be really redundant. Not to mention the VM it's running on now is too thin to support too many heavy processes.

Possible solutions:

  1. Until the cache is complete, redirect all URLs related to the model to the overview page. This is heavy-handed, because pages like the bill of materials don't make any requests for the heavy stuff, so there's no reason to block them

  2. Until caching is complete, return errors or do some kind of long polling for the spectra and datatable endpoints.

Both of these solutions also require some handling for what happens if the caching thread dies, whereas if we do northing, we occasionally use more resources than desired but everything will always fall back to on-demand and still work. Then again, if the caching thread dies, it's likely an exception or something, so the on-demand calculation is probably going to fail too...