dbt-synapse is incompatible with dbt core 0.19
MarvinSchenkel opened this issue · 6 comments
Hi,
I know this is clearly stated in the README, but I would love to use the opt-in hard deletes functionality that is available in dbt 0.19. Currently, executing dbt run with dbt 0.19 results in:
Database Error
('42000', '[42000] [Microsoft][ODBC Driver 17 for SQL Server][SQL Server]USE statement is not supported to switch between databases. Use a new connection to connect to a different Database. (104455) (SQLExecDirectW)')
I am happy to investigate and contribute, but I would love some pointers to get started :).
Cheers,
Marvin
👋 @MarvinSchenkel -- appreciate your willingness to help out! there aren't too many dbt-synapse users, but the community is growing day by day.
As far as your error goes, it's worth noting that the majority of dbt-synapse functionality comes from dbt-sqlserver.USE is supported for SQL Server and Azure SQL, but not Synapse. So whenever we see an error that has something to do with USE, it normally means that somehow dbt-sqlserver macros are being used in place of dbt-synapse's. For example just last week we fixed an exact example of this error with our v0.19.0 release with #45.
Can you call dbt --version to see what version of dbt-synapse you are using? It might also be that the wrong version of dbt-sqlserver is installed, which you should be able to check by calling pip list | grep "dbt"
The newest release of dbt-synapse is v0.19.0.1.
Then again, you could have discovered a new bug for us!
Alrighty, so this configuration is working for me:
dbt 0.18.0
dbt-bigquery 0.18.0
dbt-core 0.18.2
dbt-postgres 0.18.0
dbt-redshift 0.18.0
dbt-snowflake 0.18.0
dbt-sqlserver 0.19.0.1
dbt-synapse 0.18.1
When I try to update dbt-synapse to v0.19.0.1, I get a different error:
Running with dbt=0.19.0
Encountered an error while reading profiles:
ERROR Runtime Error
Credentials in profile "default", target "dev" invalid: Runtime Error
Could not find adapter type sqlserver!
Defined profiles:
- default
For more information on configuring profiles, please consult the dbt docs:
pip list gives me:
dbt 0.19.0
dbt-bigquery 0.19.0
dbt-core 0.19.0
dbt-postgres 0.19.0
dbt-redshift 0.19.0
dbt-snowflake 0.19.0
dbt-sqlserver 0.19.0.1
dbt-synapse 0.19.0.1
Can you give me any pointers on how to debug these errors? I am happy to help out!
Cheers,
Marvin
EDIT: Downgrading dbt-sqlserver to 0.19.0 seems to fix the error above, however, the next one I run into is:
('42000', '[42000] [Microsoft][ODBC Driver 17 for SQL Server][SQL Server]An insufficient number of arguments were supplied for the procedure or function sp_rename. (104016) (SQLExecDirectW)')
pip list:
dbt 0.19.0
dbt-bigquery 0.19.0
dbt-core 0.19.0
dbt-postgres 0.19.0
dbt-redshift 0.19.0
dbt-snowflake 0.19.0
dbt-sqlserver 0.19.0
dbt-synapse 0.19.0.1
Any hints on how to trace the sp_rename call?
Thanks for all the info! With dbt-synapse>=0.19.0, there is a breaking change that requires you to change your profiles.yml's adapter type from type=sqlserver to type=synapse
When debugging an issue that seems to change behavior between versions of the package, two good places to look are the:
CHANGELOG.md,- the GitHub Releases page, and
- any relevant Slack channel. Join #db-synapse in the dbt Slack!
The info about the breaking change is in discussed in all these places.
Thanks for all the info :). I managed to get it to work by manually installing pip packages with the correct version and changing the adapter type in my profile file.
Might be a good idea to update the README.md as well, since that still mentions dbt 0.18 ;-).
Might be a good idea to update the README.md
@MarvinSchenkel great point. As maintainers, it's hard for us to empathize with the new user experience, so we'd welcome any changes you might suggest in a PR!
Thanks for this info. Been banging my head against these several issues. Maybe also mention them in dbt Slack...