Bug: dbt-osmosis 0.13.1 is not compatible with dbt 1.8.0+
Closed this issue ยท 7 comments
Run ddbt-osmosis yaml document
in an environment with dbt-osmosis 0.13.1
, dbt-core 1.8.2
and dbt-postgres 1.8.1
and the resulting error is:
INFO ๐ Executing dbt-osmosis main.py:511
13:21:12 [WARNING]: Deprecated functionality
The `tests` config has been renamed to `data_tests`. Please see
https://docs.getdbt.com/docs/build/data-tests#new-data_tests-syntax for more information.
Traceback (most recent call last):
File "/Users/frank.tubbing/source/repos/data-warehouse-etl-dbt/venv/bin/dbt-osmosis", line 8, in <module>
sys.exit(cli())
^^^^^
File "/Users/frank.tubbing/source/repos/data-warehouse-etl-dbt/venv/lib/python3.11/site-packages/click/core.py", line 1130, in __call__
return self.main(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/frank.tubbing/source/repos/data-warehouse-etl-dbt/venv/lib/python3.11/site-packages/click/core.py", line 1055, in main
rv = self.invoke(ctx)
^^^^^^^^^^^^^^^^
File "/Users/frank.tubbing/source/repos/data-warehouse-etl-dbt/venv/lib/python3.11/site-packages/click/core.py", line 1657, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/frank.tubbing/source/repos/data-warehouse-etl-dbt/venv/lib/python3.11/site-packages/click/core.py", line 1657, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/frank.tubbing/source/repos/data-warehouse-etl-dbt/venv/lib/python3.11/site-packages/click/core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/frank.tubbing/source/repos/data-warehouse-etl-dbt/venv/lib/python3.11/site-packages/click/core.py", line 760, in invoke
return __callback(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/frank.tubbing/source/repos/data-warehouse-etl-dbt/venv/lib/python3.11/site-packages/dbt_osmosis/main.py", line 80, in wrapper
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/Users/frank.tubbing/source/repos/data-warehouse-etl-dbt/venv/lib/python3.11/site-packages/dbt_osmosis/main.py", line 513, in document
runner = DbtYamlManager(
^^^^^^^^^^^^^^^
File "/Users/frank.tubbing/source/repos/data-warehouse-etl-dbt/venv/lib/python3.11/site-packages/dbt_osmosis/core/osmosis.py", line 130, in __init__
super().__init__(target, profiles_dir, project_dir, threads, vars=vars, profile=profile)
File "/Users/frank.tubbing/source/repos/data-warehouse-etl-dbt/venv/lib/python3.11/site-packages/dbt_osmosis/vendored/dbt_core_interface/project.py", line 391, in __init__
self.parse_project(init=True)
File "/Users/frank.tubbing/source/repos/data-warehouse-etl-dbt/venv/lib/python3.11/site-packages/dbt_osmosis/vendored/dbt_core_interface/project.py", line 480, in parse_project
self.manifest = _project_parser.load()
^^^^^^^^^^^^^^^^^^^^^^
File "/Users/frank.tubbing/source/repos/data-warehouse-etl-dbt/venv/lib/python3.11/site-packages/dbt/parser/manifest.py", line 528, in load
self.check_for_spaces_in_resource_names()
File "/Users/frank.tubbing/source/repos/data-warehouse-etl-dbt/venv/lib/python3.11/site-packages/dbt/parser/manifest.py", line 641, in check_for_spaces_in_resource_names
if self.root_project.args.REQUIRE_RESOURCE_NAMES_WITHOUT_SPACES
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'DbtConfiguration' object has no attribute 'REQUIRE_RESOURCE_NAMES_WITHOUT_SPACES'
I'm experiencing the same issue with dbt-osmosis 0.13.1
, dbt-core 1.8.2
and dbt-snowflake 1.8.3
.
I was able to resolve the issue by adding the REQUIRE_RESOURCE_NAMES_WITHOUT_SPACES
field to the DbtConfiguration
class and by passing some contexts (See shinyafw@26e0b84). But I'm not certain that it's the proper fix.
We're experiencing the same issue with dbt-osmosis 0.13.1
, dbt-core 1.8.0
and dbt-bigquery 1.8.0
Getting this with dbt-osmosis 0.13.1
, dbt-core 1.8.2
and dbt-bigquery 1.8.1
, running dbt-osmosis yaml refactor <filename>.sql
INFO ๐ Executing dbt-osmosis main.py:219
Traceback (most recent call last):
File "/Users/illiayurtsiv/dev-projects/proxet/Railsreactor/gc-proxet-poc-dbt/.venv/bin/dbt-osmosis", line 8, in <module>
sys.exit(cli())
^^^^^
File "/Users/illiayurtsiv/dev-projects/proxet/Railsreactor/gc-proxet-poc-dbt/.venv/lib/python3.11/site-packages/click/core.py", line 1157, in __call__
return self.main(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/illiayurtsiv/dev-projects/proxet/Railsreactor/gc-proxet-poc-dbt/.venv/lib/python3.11/site-packages/click/core.py", line 1078, in main
rv = self.invoke(ctx)
^^^^^^^^^^^^^^^^
File "/Users/illiayurtsiv/dev-projects/proxet/Railsreactor/gc-proxet-poc-dbt/.venv/lib/python3.11/site-packages/click/core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/illiayurtsiv/dev-projects/proxet/Railsreactor/gc-proxet-poc-dbt/.venv/lib/python3.11/site-packages/click/core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/illiayurtsiv/dev-projects/proxet/Railsreactor/gc-proxet-poc-dbt/.venv/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/illiayurtsiv/dev-projects/proxet/Railsreactor/gc-proxet-poc-dbt/.venv/lib/python3.11/site-packages/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/illiayurtsiv/dev-projects/proxet/Railsreactor/gc-proxet-poc-dbt/.venv/lib/python3.11/site-packages/dbt_osmosis/main.py", line 80, in wrapper
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/Users/illiayurtsiv/dev-projects/proxet/Railsreactor/gc-proxet-poc-dbt/.venv/lib/python3.11/site-packages/dbt_osmosis/main.py", line 221, in refactor
runner = DbtYamlManager(
^^^^^^^^^^^^^^^
File "/Users/illiayurtsiv/dev-projects/proxet/Railsreactor/gc-proxet-poc-dbt/.venv/lib/python3.11/site-packages/dbt_osmosis/core/osmosis.py", line 130, in __init__
super().__init__(target, profiles_dir, project_dir, threads, vars=vars, profile=profile)
File "/Users/illiayurtsiv/dev-projects/proxet/Railsreactor/gc-proxet-poc-dbt/.venv/lib/python3.11/site-packages/dbt_osmosis/vendored/dbt_core_interface/project.py", line 391, in __init__
self.parse_project(init=True)
File "/Users/illiayurtsiv/dev-projects/proxet/Railsreactor/gc-proxet-poc-dbt/.venv/lib/python3.11/site-packages/dbt_osmosis/vendored/dbt_core_interface/project.py", line 471, in parse_project
self.config = RuntimeConfig.from_args(self.base_config)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/illiayurtsiv/dev-projects/proxet/Railsreactor/gc-proxet-poc-dbt/.venv/lib/python3.11/site-packages/dbt/config/runtime.py", line 275, in from_args
project, profile = cls.collect_parts(args)
^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/illiayurtsiv/dev-projects/proxet/Railsreactor/gc-proxet-poc-dbt/.venv/lib/python3.11/site-packages/dbt/config/runtime.py", line 254, in collect_parts
profile = cls.get_profile(
^^^^^^^^^^^^^^^^
File "/Users/illiayurtsiv/dev-projects/proxet/Railsreactor/gc-proxet-poc-dbt/.venv/lib/python3.11/site-packages/dbt/config/runtime.py", line 107, in get_profile
return load_profile(
^^^^^^^^^^^^^
File "/Users/illiayurtsiv/dev-projects/proxet/Railsreactor/gc-proxet-poc-dbt/.venv/lib/python3.11/site-packages/dbt/config/runtime.py", line 71, in load_profile
profile = Profile.render(
^^^^^^^^^^^^^^^
File "/Users/illiayurtsiv/dev-projects/proxet/Railsreactor/gc-proxet-poc-dbt/.venv/lib/python3.11/site-packages/dbt/config/profile.py", line 403, in render
return cls.from_raw_profiles(
^^^^^^^^^^^^^^^^^^^^^^
File "/Users/illiayurtsiv/dev-projects/proxet/Railsreactor/gc-proxet-poc-dbt/.venv/lib/python3.11/site-packages/dbt/config/profile.py", line 369, in from_raw_profiles
return cls.from_raw_profile_info(
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/illiayurtsiv/dev-projects/proxet/Railsreactor/gc-proxet-poc-dbt/.venv/lib/python3.11/site-packages/dbt/config/profile.py", line 315, in from_raw_profile_info
target_name, profile_data = cls.render_profile(
^^^^^^^^^^^^^^^^^^^
File "/Users/illiayurtsiv/dev-projects/proxet/Railsreactor/gc-proxet-poc-dbt/.venv/lib/python3.11/site-packages/dbt/config/profile.py", line 284, in render_profile
profile_data = renderer.render_data(raw_profile_data)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/illiayurtsiv/dev-projects/proxet/Railsreactor/gc-proxet-poc-dbt/.venv/lib/python3.11/site-packages/dbt/config/renderer.py", line 53, in render_data
return deep_map_render(self.render_entry, data)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/illiayurtsiv/dev-projects/proxet/Railsreactor/gc-proxet-poc-dbt/.venv/lib/python3.11/site-packages/dbt_common/utils/dict.py", line 129, in deep_map_render
return _deep_map_render(func, value, ())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/illiayurtsiv/dev-projects/proxet/Railsreactor/gc-proxet-poc-dbt/.venv/lib/python3.11/site-packages/dbt_common/utils/dict.py", line 94, in _deep_map_render
ret = {k: _deep_map_render(func, v, (keypath + (str(k),))) for k, v in value.items()}
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/illiayurtsiv/dev-projects/proxet/Railsreactor/gc-proxet-poc-dbt/.venv/lib/python3.11/site-packages/dbt_common/utils/dict.py", line 94, in <dictcomp>
ret = {k: _deep_map_render(func, v, (keypath + (str(k),))) for k, v in value.items()}
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/illiayurtsiv/dev-projects/proxet/Railsreactor/gc-proxet-poc-dbt/.venv/lib/python3.11/site-packages/dbt_common/utils/dict.py", line 96, in _deep_map_render
ret = func(value, keypath)
^^^^^^^^^^^^^^^^^^^^
File "/Users/illiayurtsiv/dev-projects/proxet/Railsreactor/gc-proxet-poc-dbt/.venv/lib/python3.11/site-packages/dbt/config/renderer.py", line 37, in render_entry
return self.render_value(value, keypath)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/illiayurtsiv/dev-projects/proxet/Railsreactor/gc-proxet-poc-dbt/.venv/lib/python3.11/site-packages/dbt/config/renderer.py", line 203, in render_value
raise ex
File "/Users/illiayurtsiv/dev-projects/proxet/Railsreactor/gc-proxet-poc-dbt/.venv/lib/python3.11/site-packages/dbt/config/renderer.py", line 196, in render_value
rendered = super().render_value(value, keypath)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/illiayurtsiv/dev-projects/proxet/Railsreactor/gc-proxet-poc-dbt/.venv/lib/python3.11/site-packages/dbt/config/renderer.py", line 46, in render_value
return get_rendered(value, self.context, native=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/illiayurtsiv/dev-projects/proxet/Railsreactor/gc-proxet-poc-dbt/.venv/lib/python3.11/site-packages/dbt/clients/jinja.py", line 146, in get_rendered
rendered = render_template(template, ctx, node)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/illiayurtsiv/dev-projects/proxet/Railsreactor/gc-proxet-poc-dbt/.venv/lib/python3.11/site-packages/dbt_common/clients/jinja.py", line 539, in render_template
return template.render(ctx)
^^^^^^^^^^^^^^^^^^^^
File "/Users/illiayurtsiv/dev-projects/proxet/Railsreactor/gc-proxet-poc-dbt/.venv/lib/python3.11/site-packages/dbt_common/clients/jinja.py", line 228, in render
return self.environment.handle_exception()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/illiayurtsiv/dev-projects/proxet/Railsreactor/gc-proxet-poc-dbt/.venv/lib/python3.11/site-packages/jinja2/environment.py", line 936, in handle_exception
raise rewrite_traceback_stack(source=source)
File "/Users/illiayurtsiv/dev-projects/proxet/Railsreactor/gc-proxet-poc-dbt/.venv/lib/python3.11/site-packages/dbt_common/clients/jinja.py", line 184, in quoted_native_concat
head = list(islice(nodes, 2))
^^^^^^^^^^^^^^^^^^^^^^
File "<template>", line 1, in top-level template code
File "/Users/illiayurtsiv/dev-projects/proxet/Railsreactor/gc-proxet-poc-dbt/.venv/lib/python3.11/site-packages/jinja2/sandbox.py", line 393, in call
return __context.call(__obj, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/illiayurtsiv/dev-projects/proxet/Railsreactor/gc-proxet-poc-dbt/.venv/lib/python3.11/site-packages/dbt/context/secret.py", line 32, in env_var
env = get_invocation_context().env
^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/illiayurtsiv/dev-projects/proxet/Railsreactor/gc-proxet-poc-dbt/.venv/lib/python3.11/site-packages/dbt_common/context.py", line 53, in get_invocation_context
ctx = invocation_var.get()
^^^^^^^^^^^^^^^^^^^^
LookupError: <ContextVar name='DBT_INVOCATION_CONTEXT_VAR' at 0x10744c4a0>
@illia-yurtsiv-proxet It's fixed in #160 but not yet released. I believe #164 is also required to support DBT 1.8
Thanks for the contributions @kokorin & @illia-yurtsiv-proxet -- especially the sanity tests @kokorin which were a long time coming and should make it much more straightforward to identify any obvious misses.
These have been merged and a new release cut to PyPi, 0.13.2