[CT-2936] having a folder with the same name as a macro breaks dbt docs
Opened this issue · 4 comments
Describe the bug
The dbt docs site breaks (partially) with the project/database/group staying empty the rest of the site (lineage and documentation) still works.
Looking at the browser console I can see an error like:
TypeError: Cannot set properties of undefined (setting 'foo1')
Steps To Reproduce
The bug can be reproduced by having a macro and a folder with the same name such as:
macros
├── foo
│ └── foo1.sql (contains foo1 macro)
└── foo.sql (contains foo macro)
then running
dbt docs generate
dbt docs serve
leads to a broken docs site
Expected behavior
I expect the documentation site to work if the project can be compile and run with dbt
Screenshots and log output
The output of dbt --version:
I have try with different version they all seems to have the bug
Core:
- installed: 1.4.0
- latest: 1.6.0 - Update available!
Your version of dbt-core is out of date!
You can find instructions for upgrading here:
https://docs.getdbt.com/docs/installation
Plugins:
- sqlite: 1.4.0 - Up to date!
Additional context
I have reproduce the bug while running dbt-docs in dev mode to find the origin of the error.
The issue is in the function buildProjectTree in the file project_service.js
the catalog.json and manifest.json files used to reproduce the issue can be found here: https://gist.github.com/etnnth/d2039ba410c31debb8ec3a8d844a658b
As far as I understand the error: while building the project tree it get confused because one macro as the same name as a folder and because of that it doesn't add the folder to the project tree.
In the case i rename the macro foo to foot everything works fine and the diff between the working manifest and the broken one is:
5,6c5,6
< "generated_at": "2023-08-04T08:18:31.901790Z",
< "invocation_id": "e837a159-6bfb-4c92-98a6-995193871de3",
---
> "generated_at": "2023-08-04T06:49:51.709590Z",
> "invocation_id": "859837c7-a548-4cdc-80ba-26d9fe44a31d",
75c75
< "build_path": null,
---
> "build_path": "target/run/dbt_docs_bug/models/example/my_first_dbt_model.sql",
158c158
< "build_path": null,
---
> "build_path": "target/run/dbt_docs_bug/models/example/my_second_dbt_model.sql",
256,258c256
< "macro.dbt.test_unique",
< "macro.dbt_sqlite.ref",
< "macro.dbt.get_where_subquery"
---
> "macro.dbt.test_unique"
264,268c262
< "compiled_path": "target/compiled/dbt_docs_bug/models/example/schema.yml/unique_my_first_dbt_model_id.sql",
< "compiled": true,
< "compiled_code": "\n \n \n\nselect\n id as unique_field,\n count(*) as n_records\n\nfrom main.\"my_first_dbt_model\"\nwhere id is not null\ngroup by id\nhaving count(*) > 1\n\n\n",
< "extra_ctes_injected": true,
< "extra_ctes": [],
---
> "compiled_path": null,
340,342c334
< "macro.dbt.test_not_null",
< "macro.dbt_sqlite.ref",
< "macro.dbt.get_where_subquery"
---
> "macro.dbt.test_not_null"
348,352c340
< "compiled_path": "target/compiled/dbt_docs_bug/models/example/schema.yml/not_null_my_first_dbt_model_id.sql",
< "compiled": true,
< "compiled_code": "\n \n \n\n\n\nselect id\nfrom main.\"my_first_dbt_model\"\nwhere id is null\n\n\n",
< "extra_ctes_injected": true,
< "extra_ctes": [],
---
> "compiled_path": null,
424,426c412
< "macro.dbt.test_unique",
< "macro.dbt_sqlite.ref",
< "macro.dbt.get_where_subquery"
---
> "macro.dbt.test_unique"
432,436c418
< "compiled_path": "target/compiled/dbt_docs_bug/models/example/schema.yml/unique_my_second_dbt_model_id.sql",
< "compiled": true,
< "compiled_code": "\n \n \n\nselect\n id as unique_field,\n count(*) as n_records\n\nfrom main.\"my_second_dbt_model\"\nwhere id is not null\ngroup by id\nhaving count(*) > 1\n\n\n",
< "extra_ctes_injected": true,
< "extra_ctes": [],
---
> "compiled_path": null,
508,510c490
< "macro.dbt.test_not_null",
< "macro.dbt_sqlite.ref",
< "macro.dbt.get_where_subquery"
---
> "macro.dbt.test_not_null"
516,520c496
< "compiled_path": "target/compiled/dbt_docs_bug/models/example/schema.yml/not_null_my_second_dbt_model_id.sql",
< "compiled": true,
< "compiled_code": "\n \n \n\n\n\nselect id\nfrom main.\"my_second_dbt_model\"\nwhere id is null\n\n\n",
< "extra_ctes_injected": true,
< "extra_ctes": [],
---
> "compiled_path": null,
526a503,524
> "macro.dbt_docs_bug.foo": {
> "name": "foo",
> "resource_type": "macro",
> "package_name": "dbt_docs_bug",
> "path": "macros/foo.sql",
> "original_file_path": "macros/foo.sql",
> "unique_id": "macro.dbt_docs_bug.foo",
> "macro_sql": "{% macro foo() %}\nfoo\n{% endmacro %}",
> "depends_on": {
> "macros": []
> },
> "description": "",
> "meta": {},
> "docs": {
> "show": true,
> "node_color": null
> },
> "patch_path": null,
> "arguments": [],
> "created_at": 1691131760.228975,
> "supported_languages": null
> },
7420,7441d7417
< },
< "macro.dbt_docs_bug.foot": {
< "name": "foot",
< "resource_type": "macro",
< "package_name": "dbt_docs_bug",
< "path": "macros/foo.sql",
< "original_file_path": "macros/foo.sql",
< "unique_id": "macro.dbt_docs_bug.foot",
< "macro_sql": "{% macro foot() %}\nfoo\n{% endmacro %}",
< "depends_on": {
< "macros": []
< },
< "description": "",
< "meta": {},
< "docs": {
< "show": true,
< "node_color": null
< },
< "patch_path": null,
< "arguments": [],
< "created_at": 1691137111.91506,
< "supported_languages": null
Thanks for reporting this and providing such an elegant and detailed write-up @etnnth !
Do you have a proposed fix within buildProjectTree()?
We wouldn't be able to prioritize fixing this ourselves any time soon, but we'd welcome a PR that addresses this.
I don't have a fix, but I can try to come up with something.
This issue has been marked as Stale because it has been open for 180 days with no activity. If you would like the issue to remain open, please comment on the issue or else it will be closed in 7 days.
Although we are closing this issue as stale, it's not gone forever. Issues can be reopened if there is renewed community interest. Just add a comment to notify the maintainers.