Make delay between call status check configurable
Closed this issue · 2 comments
The time between calls to internal_storage.get_call_status is hardcoded to 1 second (GET_RESULT_SLEEP_SECS):
Lines 201 to 203 in b199505
It would be useful to be able to control this so that it is the same as either
lithops.monitoring_interval(or perhaps half that value), orwait_dur_secin wait
The use case is running tests with local storage where it would be good to set it to a small value such as 0.1s so the tests run faster.
In #1338 I exposed wait_dur_sec and retries of the future methods, this can be useful in the case you manually call future.wait() or future.get_result().
However, note that in a normal lithops execution where you use the fexec.wait() method, the while loop you pointed out is never reached, since the internal monitoring system is responsible to track the available files, and only calls future.status() or future.get_result() when the status/result file is already in the storage backend.
I see that you've already discovered this in your project, but for the record in this issue: to reduce the time between checks, utilize the monitoring_interval configuration parameter, and adjust the wait_dur_sec in the fexec.wait() method.
We can discuss whether it's appropriate to automatically set the wait_dur_sec of the fexec.wait() to match or closely resemble the monitoring_interval config parameter. This way, users wouldn't need to set two different parameters for the same purpose.
We can also discuss if it s appropriate to default the monitoring_interval config parameter to 0.1 (or similar) in case of the localhost storage backend (only if it is not set by the user in the config)
Closing as the root issue is addressed by multiple PRs. WE can open a separate PR if there is a new issue or alternative requirement