fix periodic hangs
djay opened this issue · 1 comments
djay commented
Always seems to be around
[2021-10-15T09:08:02.792060+0700] ========Covid19 Timeline==========
[2021-10-15T09:08:39.134475+0700] Error downloading: inputs/json/getTimeline.json: using cache. HTTPSConnectionPool(host='covid19.th-stat.com', port=443): Max retries exceeded with url: /json/covid19v2/getTimeline.json (Caused by ReadTimeoutError("HTTPSConnectionPool(host='covid19.th-stat.com', port=443): Read timed out. (read timeout=5)"))
[2021-10-15T09:08:39.159152+0700] Importing CSV: inputs/json/deaths_all.csv
[2021-10-15T09:08:41.194199+0700] Excess Deaths: missing 2021-10
.[2021-10-15T09:08:45.228323+0700] Excess Deaths: Error in 2021-10
djay commented
haven't seen one for a day or so I think the changes I put in might have worked.
essentially with multiprocessing you lose some global state and the shared requests session no longer works or at least loses timeout settings. so ensured they got reset each time before making requests.