natcap/taskgraph

Many retries to `_execute_sqlite` should be allowed

richpsharp opened this issue · 0 comments

_execute_sqlite uses the retry module to attempt any SQLite query. This is useful when the database is locked which is always a temporary issue. However I'd put a max retries of 5 and I've got a real world case where 64 threads compete to update the database. This is all fine and expected behavior, many of them may need to wait their turn, but what happens now is TaskGraph terminates after 5 failed _execute_sqlite attempts by the threads.

I'll fix this by removing the upper limit on retries and logging relevant information that's not scary looking when the database is locked.