dbt-labs/dbt-jobs-as-code

Capability to create dbt cloud environments

venkata-varanasi opened this issue · 3 comments

Creating environments in dbt cloud is possible with Terraform as mentioned by Benoit Perigaud in here

But, Many teams are not using Terraform for any of their cloud deployments (For instance: AWS, Snowflake) so its an overkill for any of us to use it for dbt.

Capability to create dbt cloud environments with dbt-jobs-as-code along with creating jobs is nice feature to expect.

b-per commented

Thanks for raising the issue.

In your use case, what data warehouse are you using?
And ideally, for you, would those environments be created with the relevant credentials or would you want to enter the credentials in the UI?

I am asking because creating environments will be much more complex than creating jobs:

  • most environments require credentials (it depends on the data warehouse)
  • the configuration for those credentials is different for each warehouse
  • there would then potentially be the need to handle secrets as you wouldn't want to store user/pass in your repo

All those points are already taken care of by the dbt Cloud provider hence why I recommend this approach at first

We are using Snowflake. Yes the idea is to create the environments with the relevant credentials and the credentials are in AWS Secrets. We use AWS, Snowflake and dbt to create and deploy data pipelines in the production.

b-per commented

I prefer this package to be restricted to jobs.
As mentioned envs can be created from Terraform.