elementary-data/dbt-data-reliability

detection_period propery works wrong

Closed this issue · 2 comments

I'd like to complain again about the detection_period configuration property.
The following setup


 detection_period:
                period: day
                count: 1

transforms to the following statement in the elementary query which decides whether alert or not alert:

bucket_end >= 
    dateadd(day, cast('-1' as integer), cast(max_bucket_end as timestamp))

The non-strict condition means that it will alert for 2 days. But it is expected that the alert will be only one 1 day.
Now it is not possible to configure a time series test, to alert it for the current day and not for issues that were yesterday. 0 value causes test failure.

To Reproduce
setup time-series test with bucket size 1 day and detection_period 1 day

Expected behavior
With detection period of 1 day, test will not fail if start date of failed time bucket is more than 24 hours ago.
Condition which detects if test should fail have to be strict:

bucket_end >
    dateadd(day, cast('-1' as integer), cast(max_bucket_end as timestamp))

Environment (please complete the following information):

dbt package Version: 0.15.1

I suggest use here
strict >
and bucket_end > {{ elementary.edr_timeadd('day', backfill_period, 'max_bucket_end') }}

Closing to contain the discussion in the other issue.