[Bug] Dynamic project name breaks access configuration

[pgoslatara]

Is this a new bug in dbt-core?

• I believe this is a new bug in dbt-core
• I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior

When I have a dynamic project name and set access for a subdirectory via dbt_project.yml, all models in the subdirectory have an access value of protected regardless of what is specified as the access value.

Expected Behavior

The value passed to access in dbt_project.yml is respected in the parsed project.

Steps To Reproduce

1. requirements.txt:

dbt-core==1.7.14
dbt-duckdb==1.7.4

2. dbt_project.yml:

name: "{{ 'sales_' ~ target.name }}"
version: '1.0.0'
profile: 'test'

analysis-paths: ["analyses"]
test-paths: ["tests"]
seed-paths: ["seeds"]
macro-paths: ["macros"]
snapshot-paths: ["snapshots"]

clean-targets:
  - "target"
  - "dbt_packages"

models:
  "{{ 'sales_' ~ target.name }}":
    +access: public

3. profiles.yml:

test:
  target: dev
  outputs:
    dev:
      type: duckdb
      path: ":memory:"

3. ./models/test_public.sql:

select 1 as id

4. Run:

dbt build

5. Examine manifest.json:

import duckdb

conn = duckdb.connect(":memory:")
df = conn.query("""
select
    nodes."model.sales_dev.test_public".access
from './target/manifest.json'
""")
print(df)

┌───────────┐
│  access   │
│  varchar  │
├───────────┤
│ protected │
└───────────┘

The model is protected even though the configuration in dbt_project.yml specifies that it should be public.
6. We can force the output we want via:
./models/test_public.sql

{{ config(access='public') }}

select 1 as id

Run:

dbt build

import duckdb

conn = duckdb.connect(":memory:")
df = conn.query("""
select
    nodes."model.sales_dev.test_public".access
from './target/manifest.json'
""")
print(df)

┌─────────┐
│ access  │
│ varchar │
├─────────┤
│ public  │
└─────────┘

Relevant log output

No response

Environment

- OS:Ubuntu 20.04
- Python:3.11.6
- dbt:
Core:
  - installed: 1.7.14
  - latest:    1.7.14 - Up to date!

Plugins:
  - duckdb: 1.7.4 - Up to date!

Which database adapter are you using with dbt?

bigquery, other (mention it in "Additional Context")

Additional Context

• I'm setting up a dbt Mesh at a client that uses dbt-bigquery and encountered this issue. I reported this in the dbt Slack here, you can find more about our setup there.
• This issue does not arise when the name in dbt_project.yml is not dynamic, but this would prevent us from properly referencing which environment to use in our cross-project references so we do want our name to be dynamic.

[jtcohen6]

Thanks @pgoslatara!

this would prevent us from properly referencing which environment to use in our cross-project references

I just responded in slack to get a bit more context. My goal is to support the pattern of environment isolation you're after, without the need for dynamic project names.

[jtcohen6]

@pgoslatara Summarizing from Slack thread: It our intent to support cross-project references where 1 project == 1 project. Each project name must be unique within the dbt Cloud account. We don't intend to support patterns where multiple dbt Cloud projects have the same project name.

To that end, we are working hard to remove the historical limitations that required customers to "duplicate" projects in dbt Cloud — Staging environments for data isolation (beta), env-level permissions and env-level data warehouse connections (coming very soon).

---

In the meantime, because dbt does not support dynamic project names within FQN-based configuration, I'm aware of a partial workaround: you can define the same project config multiple times, once for each known project name, and use yaml anchors to keep it DRY (s/o to the customer who came up with this, iykyk @kbrock91):

models:
  sales_prod: &model_config
    +access: public
    subfolder:
      +materialized: table
      # etc
  sales_dev: *model-config

That would still require the user to maintain different dependencies.yml in downstream projects, for development versus production, because these files do not support Jinja rendering for conditional project names. I'm not sure how you would facilitate this workflow, to avoid accidentally committing the update as code changes move from dev into prod.

# dependencies.yml
projects:
  - name: sales_prod  # need to switch this to 'sales_dev' in development

[jtcohen6]

I am going to close this as wontfix, given the longer-term pattern is "1 project == 1 project"