duckdb/dbt-duckdb

external_root not being recognized

Closed this issue · 1 comments

This is most definitely a misunderstanding on my part, but I cannot seem to get the profile setting, external_root working.

If I do not specify a location the model is materialized relative to the path of my project. When I specify a location on the model it is materialized to S3 as expected.

I have tried passing external_root unquoted, quoted and as an env_var.

requirements.txt

duckdb==0.9.0
dbt-duckdb==1.6.1

profile.yml

  outputs:
    aws:
      extensions:
      - httpfs
      - parquet
      module_paths:
      - plugins
      path: "/tmp/duck.db"
      threads: 1
      external_root: "s3://datalake-prod"
      settings:
        s3_region: us-west-2
        s3_access_key_id: '{{ env_var("AWS_ACCESS_KEY_ID") }}'
        s3_secret_access_key: '{{ env_var("AWS_SECRET_ACCESS_KEY") }}'
        s3_session_token: '{{ env_var("AWS_SESSION_TOKEN") }}'
      type: duckdb
  target: aws

model.sql

{{  
  config(
    materialized='external',
    options={
      'partition_by': '(year, month, day)',
      'overwrite_or_ignore': 1,
      'filename_pattern': '"{{this.table}}_{i}"'
    },
  ) 
}}

My mistake, my development docker container was looking at a different profile.yml. Closing.