How does mock_source() mock the source with <database>.<schema>?
ACeethoven opened this issue · 2 comments
Hi team. First, thanks for building this tool!
I ran into an issue when using mock_source()
, it seems that it didn't mock the source very well as it should be in my bigquery project. I have a source yaml file and it contains something like:
version: 2
sources:
- name: test_name
database: test_gcp_name
schema: test_dataset_name
tables:
- name: test_table_name
And I use source('test_name', 'test_table_name')
in my models to access my bigquery table, my model looks like:
...
WITH s AS (
...
source('test_name', 'test_table_name')
...
)
...
During my testing, I found that when I tried to use dbt_unit_testing.mock_source('test_name', 'test_table_name')
to test my model, it did not replace the table name used in my model, which, from source('test_name', 'test_table_name')
, is test_gcp_name.test_dataset_name.test_table_name
.
I was debugging this with complete raw SQL it generated, and found that dbt-unit-testing.mock_source()
did help create a table named test_table_name
, but this table name was not used to replace the one in my model. I also tried with use_qualified_sources
option, but it did help much.
I wonder if I missed anything here or did not use mock_source
in a correct way or there is something I should set?
Thanks in advance!
OK nvm. I figured that we cannot use dbt native source
but instead we should use dbt-unit-testing.source(...)
to let the source be replaced. Also the tags value have to be unit-test
(
dbt-unit-testing/macros/overrides.sql
Lines 35 to 37 in e54329c
Resolved.
Hello! Have same issue. Could you please describe more verbose how to fix this issue?
- How to mock source entirely to skip searching for it in the database?
- How to use any database.source that are not equal to target-database.target-source?
for example, I have this sources.yml
sources:
- name: ref
database: "{{ var('dbt_ref_database') }}"
schema: "{{ var('dbt_ref_schema') }}"
tables:
- name: ref_tenant_classifier_config
and this part inside unit-test.sql
{% call dbt_unit_testing.mock_source("ref", "ref_tenant_classifier_config") %}
col1,col2
val1,val2
{% endcall %}
It yields at me that the table doesn't exist in the target-database.target-schema ignoring the vars() in the shema.yml.
I have overridden macros:
{% macro generate_schema_name(custom_schema_name, node) -%}
{%- set default_schema = target.schema -%}
{%- if custom_schema_name is none -%} {{ default_schema }}
{%- else -%} {{ custom_schema_name | trim }}
{%- endif -%}
{%- endmacro %}
{% macro ref() %} {{ return(dbt_unit_testing.ref(*varargs, **kwargs)) }} {% endmacro %}
{% macro source() %}
{{ return(dbt_unit_testing.source(*varargs, **kwargs)) }}
{% endmacro %}
Also I have specified all columns but it still goes to the database to check the table structure and fails.