Unquoted strings in Jinja macro leads to bad formatting
Closed this issue · 1 comments
Describe the bug
Unquoted strings within dbt Jinja macros are parsed improperly, and lead to unexpected/breaking changes
To Reproduce
Run sqlfmt on a macro
{% macro generate_external_location(lake_zone, prefix) -%}
s3://data-lake-{{ lake_zone }}-{{ target.name }}/{{ prefix }}/foo/bar
{%- endmacro %}
Expected behavior
No changes are made
Actual behavior
On sqlfmt.com, strings are split, leading to a bad S3 URI.
{% macro generate_external_location(lake_zone, prefix) -%}
s3: / / data - lake -{{ lake_zone }}-{{ target.name }}/{{ prefix }}/ foo / bar
{%- endmacro %}
For some reason, I am getting a different (but still breaking) result locally
{% macro generate_external_location(lake_zone, prefix) -%}
s3: -- data-lake-{{ lake_zone }}-{{ target.name }}/{{ prefix }}/foo/bar
{%- endmacro %}
Additional context
sqlfmt, version 0.20.0
In both cases, it seems that the tokens are being read literally, i.e. //
is read as a SQL comment, and is transformed into --
.
The result of the generate_external_location macro feeds into a dbt-athena config block, like so:
{{
config(
materialized='incremental',
incremental_strategy='insert_overwrite',
external_location=generate_external_location("bucket", "object=foo"),
partitioned_by=['batch_date']
)
}}
Macros can template anything, but we assume they are templating sql. An S3 URI is not valid SQL. Just place single quotes around it or --fmt: off on a line above the URI.
In snowflake //
is a comment marker; your second example is from a more recent version of sqlfmt, where we standardize that comment marker to --