djay/covidthailand

fix periodic hangs

djay opened this issue · 1 comments

djay commented

Always seems to be around

[2021-10-15T09:08:02.792060+0700] ========Covid19 Timeline==========
[2021-10-15T09:08:39.134475+0700] Error downloading: inputs/json/getTimeline.json: using cache. HTTPSConnectionPool(host='covid19.th-stat.com', port=443): Max retries exceeded with url: /json/covid19v2/getTimeline.json (Caused by ReadTimeoutError("HTTPSConnectionPool(host='covid19.th-stat.com', port=443): Read timed out. (read timeout=5)"))
[2021-10-15T09:08:39.159152+0700] Importing CSV: inputs/json/deaths_all.csv
[2021-10-15T09:08:41.194199+0700] Excess Deaths: missing 2021-10
.[2021-10-15T09:08:45.228323+0700] Excess Deaths: Error in 2021-10
djay commented

haven't seen one for a day or so I think the changes I put in might have worked.
essentially with multiprocessing you lose some global state and the shared requests session no longer works or at least loses timeout settings. so ensured they got reset each time before making requests.