datnguye/dbterd

[BUG] Ephemeral Models have {"catalog": null} using Discovery API - Column Error

Closed this issue ยท 3 comments

Describe the bug
Hey ๐Ÿ’ฏ
I've been testing the Discovery API solution, but it failed because of my ephemeral models.

Graph example:

               "node": {
                "uniqueId": "model.my_project.my_ephemeral_model",
                "name": "my_ephemeral_model",
                "description": "",
                "database": "my_db",
                "schema": "my_schema",
                "alias": "my_ephemeral_model",
                "catalog": null
              }

In base.py def get_table_from_metadata if expects to get columns from the catalog:

    for column in model_metadata.get("node", {}).get("catalog", {}).get("columns", []):
        table.columns.append(
            Column(
                name=column.get("name", "").lower(),
                data_type=column.get("type", "").lower(),
                description=column.get("description", ""),
            )
        )

To Reproduce
Steps to reproduce the behavior:

  1. Create a dbt ephemeral model
  2. Rerun your dbt Cloud job having docs generated
  3. run dbterd run-metadata

Expected behavior
It should handle ephemeral models in any way. I just put the loop into a if-clause, but I'm sure you'll find a good solution being aware of the full application structure.

Best,
Marvin

Thanks @marvingeerken! Let me check it out and get back to this soon

@marvingeerken It will be fixed in above mentioned PR. Thanks for finding it out ๐Ÿ™Œ

Fix should be available in 1.10.1, also be forward to v1.11.0 now๐ŸŽ‰