datafold/data-diff

Exception using `data-diff --dbt` on dbt-core 1.4 versions

dlawin opened this issue · 0 comments

dlawin commented

Describe the bug
When I run data-diff with a dbt-core 1.4.X package installed, I see an exception

To reproduce the issue:

✗ pip install 'dbt-snowflake<1.5,>=1.4'
✗ dbt --version
Core:
  - installed: 1.4.9
✗ data-diff --version
Traceback (most recent call last):
  File "/Users/dan/repos/dbt-analytics/.venv/bin/data-diff", line 5, in <module>
    from data_diff.__main__ import main
  File "/Users/dan/repos/dbt-analytics/.venv/lib/python3.11/site-packages/data_diff/__main__.py", line 19, in <module>
    from data_diff.dbt import dbt_diff
  File "/Users/dan/repos/dbt-analytics/.venv/lib/python3.11/site-packages/data_diff/dbt.py", line 20, in <module>
    from data_diff.cloud import DatafoldAPI, TCloudApiDataDiff, TCloudApiOrgMeta
  File "/Users/dan/repos/dbt-analytics/.venv/lib/python3.11/site-packages/data_diff/cloud/__init__.py", line 2, in <module>
    from data_diff.cloud.data_source import get_or_create_data_source
  File "/Users/dan/repos/dbt-analytics/.venv/lib/python3.11/site-packages/data_diff/cloud/data_source.py", line 18, in <module>
    from data_diff.dbt_parser import DbtParser
  File "/Users/dan/repos/dbt-analytics/.venv/lib/python3.11/site-packages/data_diff/dbt_parser.py", line 12, in <module>
    from dbt.config.renderer import ProfileRenderer
  File "/Users/dan/repos/dbt-analytics/.venv/lib/python3.11/site-packages/dbt/config/__init__.py", line 2, in <module>
    from .profile import Profile, read_user_config  # noqa
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/dan/repos/dbt-analytics/.venv/lib/python3.11/site-packages/dbt/config/profile.py", line 8, in <module>
    from dbt.clients.system import load_file_contents
  File "/Users/dan/repos/dbt-analytics/.venv/lib/python3.11/site-packages/dbt/clients/system.py", line 17, in <module>
    from dbt.events.functions import fire_event
  File "/Users/dan/repos/dbt-analytics/.venv/lib/python3.11/site-packages/dbt/events/__init__.py", line 1, in <module>
    from .adapter_endpoint import AdapterLogger  # noqa: F401
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/dan/repos/dbt-analytics/.venv/lib/python3.11/site-packages/dbt/events/adapter_endpoint.py", line 3, in <module>
    from dbt.events.functions import fire_event
  File "/Users/dan/repos/dbt-analytics/.venv/lib/python3.11/site-packages/dbt/events/functions.py", line 4, in <module>
    from dbt.events.eventmgr import EventManager, LoggerConfig, LineFormat, NoFilter
  File "/Users/dan/repos/dbt-analytics/.venv/lib/python3.11/site-packages/dbt/events/eventmgr.py", line 13, in <module>
    import dbt.utils
  File "/Users/dan/repos/dbt-analytics/.venv/lib/python3.11/site-packages/dbt/utils.py", line 42, in <module>
    import dbt.exceptions
  File "/Users/dan/repos/dbt-analytics/.venv/lib/python3.11/site-packages/dbt/exceptions.py", line 7, in <module>
    from dbt.internal_deprecations import deprecated
  File "/Users/dan/repos/dbt-analytics/.venv/lib/python3.11/site-packages/dbt/internal_deprecations.py", line 4, in <module>
    from dbt.events.functions import warn_or_error
ImportError: cannot import name 'warn_or_error' from partially initialized module 'dbt.events.functions' (most likely due to a circular import) (/Users/dan/repos/dbt-analytics/.venv/lib/python3.11/site-packages/dbt/events/functions.py)

To workaround the issue:

  • Update to a dbt-core version >= 1.5
    • Downgrading may also work but have not tested

Workaround isn't going to be viable for some folks