[Regression] Jinja templating not working in dependencies.yml
Closed this issue · 4 comments
Is this a regression in a recent version of dbt-core?
- I believe this is a regression in dbt-core functionality
- I have searched the existing issues, and I could not find an existing issue for this regression
Current Behavior
packages.yml
file supports jinja templating, at least to read environment variables with env_var()
.
For instance:
packages:
- package: dbt-labs/dbt_utils
version: '{{ env_var("DBT_UTILS_VERSION", "1.3.0") }}'
This does not work anymore when renaming the file to dependencies.yml
and this will be an issue in the future.
We use this to pull private dbt packages from gitlab repositories using a syntax like that:
packages:
- package: dbt-labs/dbt_utils
version: 1.3.0
- git: "https://{{ ('gitlab-ci-token:' ~ env_var('CI_JOB_TOKEN') ~ '@') if env_var('CI_JOB_TOKEN', '') }}gitlab.com/path/to/my/package"
revision: 1.1.1
Expected/Previous Behavior
Jinja templating should be re-enabled in the file dependencies.yml
Steps To Reproduce
Simply add jinja templating trying to read an existing (or not) environment variable in dependencies.yml and you'll get a syntax error.
Relevant log output
No response
Environment
- OS: MacOS
- Python: 3.11.9
- dbt (working version): whatever the version if you're using packages.yml
- dbt (regression version): whatever the version if you're switching to dependencies.yml
Which database adapter are you using with dbt?
bigquery
Additional Context
No response
Hey @H-Max ! I see what you are saying about Jinja templating not being supported in dependencies.yml
and how this affects using environment variables for private packages within dependencies.yml
.
The solution for your case:
- Keep any package dependencies within your existing
packages.yml
file - Add any project dependencies to a brand-new
dependencies.yml
file.
Relevant documentation
We have this documented here, but it is easy to miss:
Next steps
We intentionally chose for dependencies.yml
to not include Jinja rendering, so I'm going to close this as not a bug or a regression.
Rather, you can keep any dbt packages that need Jinja rendering within your packages.yml
file and use it alongside a new dependencies.yml
file.
Thanks Doug for the heads up to the documentation. I did miss it.
I really thought I read somewhere that it was a good practice to move everything to dependencies.yml, and I wonder if there wasn't a time when there was a warning about packages.yml depreciation, but I am may be wrong about that.
Thanks again for the quick reply on the issue !
I really thought I read somewhere that it was a good practice to move everything to dependencies.yml, and I wonder if there wasn't a time when there was a warning about packages.yml depreciation, but I am may be wrong about that.
You have a good memory @H-Max 🧠
Our original guidance was to move everything to dependencies.yml
. The problem is that not everyone can do that. So we updated our guidance in dbt-labs/docs.getdbt.com#6129 / dbt-labs/docs.getdbt.com#6171.