Make vars templated_fields

[tomsej]

Hi thanks for this repo. Could you please make a vars a templated field? https://airflow.apache.org/docs/stable/concepts.html#jinja-templating

[andrewrjones]

Hi @tomsej,

Thanks for the suggestion. We can certainly do this, and it would make a lot of sense. We'll try to get to this in the next day or two!

Thanks,
Andrew

[ee07dazn]

Hi Guys, thanks for this repo. Any update on the issue? I am keen to use {{ ds }} from airflow into a query in one of my models in dbt. And i believe the resolution to this issue will solve my problem.

Update: Thinking further, i believe the {{ ds }} should have already worked as it is deeply embedded within the BaseOperator, which is being used to build the DBTXOperators. Any thoughts ?

This is how I tried to use it :

SELECT "id"
        FROM {{ source('PCMS', 'article') }}
        WHERE BATCH_DATE = '{{ ds }}'

The actual sql that got run was with "BATCH_DATE = ' ' "

Update (another approach): I tried to import DbtRunOperator into my custom class like this :

from airflow_dbt.operators.dbt_operator import DbtRunOperator
from airflow.utils.decorators import apply_defaults

class EDMDbtRunOperator(DbtRunOperator):

    template_fields = ('vars',)

    @apply_defaults
    def __init__(self, vars=None, *args, **kwargs):
        self.vars = vars
        super(EDMDbtRunOperator, self).__init__(*args, vars=self.vars, **kwargs)

Then in my dag passed the following task:

dbt_run = EDMDbtRunOperator(task_id=dag.partner,
                             dir=dag.default_args['dbt_dir'],
                             models='content_dimension',
                             vars={'batch_date': '{{ ds }}' },
                             dag=dag)

However, the actual run ended up like this :

"dbt run --vars {"batch_date": "{{ ds }}"} --models content_dimension" instead of the substitution for 'ds', which causes the same issue of running "BATCH_DATE = ' ' " .

@andrewrjones , @tomsej Any idea on what i am doing wrong here ?