Velir/dbt-ga4

combine_property_data does not drop old intraday tables

Closed this issue · 0 comments

When using the multi-property + streaming import feature introduced with #187, then the combined dataset accumulates intraday tables over time. As soon as for the intraday table analytics_xxxxx.events_intraday_yyyyyyyy its corresponding daily table analytics_xxxxx.events_yyyyyyyy becomes available, combine_property_data will copy it but leave the old <combined_dataset>.events_intraday_yyyyyyyyxxxxx behind.
This ultimately leads to duplicated data in base_ga4__events that is not sufficiently removed by the final qualify row_number() .... = 1 statement.