[CT-2803] [CT-2801] [Feature] add Macros to Lineage Graph (DAG)
ym-onprem opened this issue · 3 comments
Is this your first time submitting a feature request?
- I have read the expectations for open source contributors
- I have searched the existing issues, and I could not find an existing issue for this feature
- I am requesting a straightforward extension of existing dbt functionality, rather than a Big Idea better suited to a discussion
Describe the feature
DAGs are great for many reasons. Currently the DAG in the generated docs does not include macros, making it difficult to navigate through a maze of macros, which are often called by other macros, models, and other resources. Maybe we start with just the custom macros in the macros folder and go from there.
Existing resources in the lineage graph (1.5.1)
Mock-up of the feature... :)
Describe alternatives you've considered
Jump from file to file and read the code for each macro to determine the dependencies.
Who will this benefit?
Anyone who deals with macros :)
Are you interested in contributing this feature?
No response
Anything else?
No response
@ym-onprem Neat idea! Are you thinking that this would help with digging through complex webs of macros-calling-macros? Seeing all downstream resources which depend on a given macro — and, if that macro were to change, would be selected by state:modified?
Everything that's needed here should already be present in the manifest — every resource has a depends_on.macros attribute, including macros (if they call other macros). Note that there can be some subtle differences based on when a manifest is produced (dbt-labs/dbt-core#5079): a post-parse/compile manifest will include only the macros called within a model's raw_code, whereas a post-run/build manifest will also include macros called within the materialization.
In my view, it's still appropriate for macro dependencies to be hidden from the standard DAG visualization. But perhaps as an additional layer, or another DAG viz...?
I'm going to transfer this over to the dbt-docs repo, since that's where the changes would need to happen. Candidly, our team isn't able to prioritize new feature development in dbt-docs, so this isn't something we'd be picking up ourselves.
This issue has been marked as Stale because it has been open for 180 days with no activity. If you would like the issue to remain open, please comment on the issue or else it will be closed in 7 days.
Although we are closing this issue as stale, it's not gone forever. Issues can be reopened if there is renewed community interest. Just add a comment to notify the maintainers.