Add handling for datetime aware start time in PeriodicExecutionTimeMonitor
Closed this issue · 1 comments
further-reading commented
In the current version of scrapy, the code below breaks due to spider start time being timezone aware
https://github.com/scrapinghub/spidermon/blob/master/spidermon/contrib/scrapy/monitors/monitors.py#L497
now = datetime.datetime.utcnow()
start_time = self.data.stats.get("start_time")
duration = now - start_time
It currently throws the following error:
TypeError: can't subtract offset-native and offset-aware datetimes
Backwards compatible Fix: add try/except to add utc timezone to now timestamp when error thrown.