dbt found two macros named "synapse__drop_relation" in the project
brandonlockhart opened this issue · 15 comments
Hello,
With versions dbt-core==1.0.0, dbt-sqlserver==1.0.0, dbt-synapse==1.0.1, when I run a dbt command such as dbt snapshot, I get the error
20:32:03 Running with dbt=1.0.0
20:32:04 Partial parse save file not found. Starting full parse.
20:32:04 Encountered an error:
Compilation Error
dbt found two macros named "synapse__drop_relation" in the project
"dbt_synapse".
To fix this error, rename or remove one of the following macros:
- macros/adapters/relation.sql
- macros/adapters.sql
I do not get this error with versions dbt-core==0.20.2, dbt-sqlserver==0.20.1, dbt-synapse==0.20.0. I'm using Python version 3.9.9.
@brandonlockhart, sorry about that! we're also experiencing this bug with dbt-synapse in dbt-msft/tsql-utils#76. the tests fail with same error message! We hope to find a solution soon, though it likely won't be until after the holidays as most of us will be OOO. hang tight! cc: @jtcohen6
until then stick with v0.21.0!
@leahwicz @jtcohen6 have y'all seen this issue before? this is a showstopping bug in the 1.0.0 release I made before the break. there's a link to a CircleCi failure above. there aren't duplicate macros, and the macro that says it's duplicated changes from commit to commit, so I'm rather confused.
also I'd like to get #77 fixed so my name doesn't live in peoples' console and manifest anymore.... 🤒
@swanderz Given the error message, and that #72 moved this macro from macros/adapters.sql to macros/adapters/relation.sql... If you install dbt-synapse pre-v1.0, and then upgrade to v1.0, is it possible that the old file keeps "hanging out" in the source installation?
I quickly tried replicating this locally, and I wasn't able to
I was able to reproduce the error both on macos as well as Windows (python3.8). When I reviewed the RECORD file in site-packages/dbt_synapse-1.0.1.dist-info in my python environment I could verify that indeed during the installation files such as dbt/include/synapse/macros/adapters.sql and dbt/include/synapse/macros/materializations/seed/seed.sql were written, which lead to duplications (files that are, curiously, not present in the include folder of the 1.0.1 version of the repository).
I was seemingly able to hotfix this by adapting the setup.py by changing packages=find_namespace_packages(include=['dbt', 'dbt.*']), to packages=find_packages(), since I was wondering whether this had something to do with mixing native and pkgutil-style namespaces (see docs). But honestly, I am very puzzled where those files come from, and I am aware that the setup.py seemingly works for the dbt-sqlserver adapter.
@dschoenleber thanks for digging into this! You got farther than I did. @jtcohen6's question is does this behavior happen when upgrading to 1.0.0 and maybe the old adapters.sql file stays around? asked another way, if you make a fresh python environment and install dbt-synapse 1.0.0 does the error still happen?
I'd also like to fix this in the next few days, sounds like you found a hotfix that I'll try out on my end as well. Cheers!
@dataders Your're welcome! Yes, the behavior I described occurred with a freshly initialized environment with no prior dbt (or dbt-synapse) installation around, so the behavior is triggered by the installation of dbt-synapse 1.0.1. This makes it so curious.
I can replicate this in my devcontainer with a clean install. Also, the name of the duplicate macro is not constant. It's almost always a different one each time you run dbt compile.
Looks like the files were left on the machine that created the package, because when you just recreate the wheel, they're not inside the archive. I attached a wheel file created from the current version of the source where it doesn't show that behaviour.
@dataders Maybe we should setup automatic publishing to pypi when a new tag is created to avoid such issues? Anyway, if you create a new release 1.0.2, then make sure you have a clean version of the repo locally and recreate the wheel, then we're good.
@sdebruyn This would also explain my observations. (If you install via pip install git+https://github.com/dbt-msft/dbt-synapse.git the additional files are also not included.)
Thanks so much for digging into it!
No more errors for me either with 1.0.2b1. Thanks @dataders !
When can we look forward to the official release of 1.0.2? :)
1.0.2 has been released :)