godatadriven/pytest-dbt-core

[Feature] Support dbt 'is_incremental' in pytest-dbt-core

Opened this issue · 1 comments

Is this your first time opening an issue?

Describe the Feature

dbt adds the 'this' somehow in the is_incremental macro:
https://github.com/dbt-labs/dbt-core/blob/main/core/dbt/include/global_project/macros/materializations/models/incremental/is_incremental.sql
is_incremental.sql

{% macro is_incremental() %}
{#-- do not run introspective queries in parsing #}
{% if not execute %}
{{ return(False) }}

Describe alternatives you've considered

I would like to test the following macro that uses 'is_incremental' and 'this' argument

{% macro get_table_max_id(where_clause = '', id_column = '"id"') %}
    {% if is_incremental() %}
        {% set max_id_query %}
        select coalesce(max({{ id_column }}), 0) from {{ this }}
        {{ where_clause }}
        {% endset %}
        {% set max_id_res = run_query(max_id_query) %}
    {%endif%}
    {{ return(max_id) }}
{% endmacro %}

following error:
def raise_compiler_error(msg, node=None) -> NoReturn:

  raise CompilationException(msg, node)

E dbt.exceptions.CompilationException: Compilation Error in macro is_incremental (macros/materializations/models/incremental/is_incremental.sql)
E 'this' is undefined
E
E > in macro get_table_max_id (macros/unioner.sql)
E > called by macro test_table_max_id (macros/test_macro.sql)
E > called by macro is_incremental (macros/materializations/models/incremental/is_incremental.sql)

Who will this benefit?

All dbt users since is_incremental is very popular

Are you interested in contributing this feature?

No response

Anything else?

No response

Hi @danaeder, thanks for opening the issue. I see the this is missing; dbt adds this somehow; this package does not (yet).

Could you try the following for me?

{% macro get_table_max_id(where_clause = '', id_column = '"id"') %}
        {% set max_id_query %}
        select coalesce(max({{ id_column }}), 0) from {{ this }}
        {{ where_clause }}
        {% endset %}
        {% set max_id_res = run_query(max_id_query) %}
    {{ return(max_id) }}
{% endmacro %}

What happens with the this after the from (third line)? Is that compiled correctly now?

And, something that might unblock you quicker, is it possible to use the is_incremental outside the macro. I expect that you have something like:

SELECT * 
FROM my_table 
{{ get_table_max_id() }}

What happens if you move the is_incremental out of the macro:

SELECT * 
FROM my_table 
{% if is_incremental() %}
{{ get_table_max_id() }}
{%endif%}