EqualExperts/dbt-unit-testing

How does mock_source() mock the source with <database>.<schema>?

ACeethoven opened this issue · 2 comments

Hi team. First, thanks for building this tool!

I ran into an issue when using mock_source(), it seems that it didn't mock the source very well as it should be in my bigquery project. I have a source yaml file and it contains something like:

version: 2

sources:
    - name: test_name
      database: test_gcp_name
      schema: test_dataset_name
      tables:
      - name: test_table_name

And I use source('test_name', 'test_table_name') in my models to access my bigquery table, my model looks like:

...
WITH s AS (
    ...
    source('test_name', 'test_table_name')
    ...
)
...

During my testing, I found that when I tried to use dbt_unit_testing.mock_source('test_name', 'test_table_name') to test my model, it did not replace the table name used in my model, which, from source('test_name', 'test_table_name'), is test_gcp_name.test_dataset_name.test_table_name.

I was debugging this with complete raw SQL it generated, and found that dbt-unit-testing.mock_source() did help create a table named test_table_name, but this table name was not used to replace the one in my model. I also tried with use_qualified_sources option, but it did help much.

I wonder if I missed anything here or did not use mock_source in a correct way or there is something I should set?
Thanks in advance!

OK nvm. I figured that we cannot use dbt native source but instead we should use dbt-unit-testing.source(...) to let the source be replaced. Also the tags value have to be unit-test (

{% macro running_unit_test() %}
{{ return ('unit-test' in config.get('tags', {})) }}
{% endmacro %}
).

Resolved.

Hello! Have same issue. Could you please describe more verbose how to fix this issue?

  1. How to mock source entirely to skip searching for it in the database?
  2. How to use any database.source that are not equal to target-database.target-source?

for example, I have this sources.yml

sources:
  - name: ref
    database: "{{ var('dbt_ref_database') }}"
    schema: "{{ var('dbt_ref_schema') }}"
    tables:
      - name: ref_tenant_classifier_config

and this part inside unit-test.sql

{% call dbt_unit_testing.mock_source("ref", "ref_tenant_classifier_config") %}
    col1,col2
    val1,val2
    {% endcall %}

It yields at me that the table doesn't exist in the target-database.target-schema ignoring the vars() in the shema.yml.

I have overridden macros:

{% macro generate_schema_name(custom_schema_name, node) -%}
    {%- set default_schema = target.schema -%}
    {%- if custom_schema_name is none -%} {{ default_schema }}
    {%- else -%} {{ custom_schema_name | trim }}
    {%- endif -%}
{%- endmacro %}

{% macro ref() %} {{ return(dbt_unit_testing.ref(*varargs, **kwargs)) }} {% endmacro %}

{% macro source() %}
    {{ return(dbt_unit_testing.source(*varargs, **kwargs)) }}
{% endmacro %}

Also I have specified all columns but it still goes to the database to check the table structure and fails.