mozanalysis can't handle `submission_date_column`s that are SQL snippets
Opened this issue · 1 comments
scholtzan commented
The problem is occurring here:
As long as a data sources submission_date_column
is only set to a field name everything is working fine, but as soon as SQL is used the generated query is invalid due to the alias. ds.{submission_date}
cannot be transformed to ds.DATE(submission_timestamp)
, for example
jaredsnyder commented
The code below is useful to recreating this bug (assuming I understand the issue)
from mozanalysis.metrics import DataSource
data_source_definition = config_collection.get_data_source_definition(
data_source_slug, app_name
)
if data_source_definition is None:
raise Exception(
f"Could not find definition for data source {data_source_slug}"
)
clients_daily = DataSource(
name=data_source_definition.name,
from_expr=data_source_definition.from_expression,
client_id_column=data_source_definition.client_id_column,
submission_date_column="Date(submission_timestamp_min)",
experiments_column_type=None
if data_source_definition.experiments_column_type == "none"
else data_source_definition.experiments_column_type,
default_dataset=data_source_definition.default_dataset,
)
from mozanalysis.metrics import DataSource, Metric, agg_any, agg_sum
qcdou = Metric(
name="qcdou",
data_source=clients_daily,
select_expr="""COUNTIF(
active_hours_sum > 0 AND
scalar_parent_browser_engagement_total_uri_count_normal_and_private_mode_sum > 0
)""",
)
df = test_target.get_single_window_data(
bq_context=bq_context,
metric_list=[qcdou],
target_list=[weekday_regular_us]
)