Improvements to error logging
jeanpaulrsoucy opened this issue · 1 comments
jeanpaulrsoucy commented
- Ensure failed downloads are explicitly logged in an easy-to-understand way
-
When an error for a particular dataset/value is logged, should have a script that checks if a) there is already a value defined for today (and whether than value is automated or manual) and b) if there is a value, if that value is different from the previous day's value (e.g., maybe the previously logged value was pre-update). This will help make the error log more actionable.
Note that these improvements may require more than a simple sink function as is currently implemented.
jeanpaulrsoucy commented
See ccodwg/Covid19CanadaDataProcess@09bacd5 and ccodwg/Covid19CanadaETL@4d0e314 for better failed download error logging.