Goal

Run DBT jobs that are observed with Prefect and keep credential in your environemnt (don't a Prefect Cloud Credentials Block). Generalizes to other integrations.

Could use Docker, K8s, ECS, or whatever infrastructure type you like, just make sure your environment variable is injected.

Steps

  1. Install prefect-dbt
  2. Register prefect-dbt module blocks
  3. Get API key from DBT Cloud
  4. Create work pool
  5. Create flow code
  6. Create deployment code (specify work pool)
  7. Could use a SecretVolume