[BUG] Ephemeral Models have {"catalog": null} using Discovery API - Column Error
Closed this issue ยท 3 comments
marvingeerken commented
Describe the bug
Hey ๐ฏ
I've been testing the Discovery API solution, but it failed because of my ephemeral models.
Graph example:
"node": {
"uniqueId": "model.my_project.my_ephemeral_model",
"name": "my_ephemeral_model",
"description": "",
"database": "my_db",
"schema": "my_schema",
"alias": "my_ephemeral_model",
"catalog": null
}
In base.py def get_table_from_metadata if expects to get columns from the catalog:
for column in model_metadata.get("node", {}).get("catalog", {}).get("columns", []):
table.columns.append(
Column(
name=column.get("name", "").lower(),
data_type=column.get("type", "").lower(),
description=column.get("description", ""),
)
)
To Reproduce
Steps to reproduce the behavior:
- Create a dbt ephemeral model
- Rerun your dbt Cloud job having docs generated
- run dbterd run-metadata
Expected behavior
It should handle ephemeral models in any way. I just put the loop into a if-clause, but I'm sure you'll find a good solution being aware of the full application structure.
Best,
Marvin
datnguye commented
Thanks @marvingeerken! Let me check it out and get back to this soon
datnguye commented
@marvingeerken It will be fixed in above mentioned PR. Thanks for finding it out ๐