[Bug] Using Data Source name in Replace Columns statement doesn't work
selmanmoon opened this issue · 0 comments
Is this a new bug in dbt-athena?
- I believe this is a new bug in dbt-athena
- I have searched the existing issues, and I could not find an existing issue for this bug
Current Behavior
Hi,
I believe this is more likely an issue with Athena therefore I contacted with internal team as well, however until there is an update from there maybe we can slightly update the repo to make this work.
Problem:
When running the following statement in AWS Athena. When I include AwsDataCatalog
in the query, it fails, but it works fine if I drop it.
ALTER TABLE
AwsDataCatalog
.default
.selman_schema_test
REPLACE COLUMNS (column1 date, column2 string);
This statement works without AwsDataCatalog
:
ALTER TABLE
default
.selman_schema_test
REPLACE COLUMNS (column1 date, column2 string);
I'm using below option to test schema changes on parquet files with dbt-athena;
+on_schema_change: "sync_all_columns"
Therefore when there is a schema change, dbt-athena plugin generates replace columns statement as the first query above and it fails. maybe we can update the code to exclude datasource when generating SQL statement, if that's possible.
Any suggestion is appreciated, thanks.
Expected Behavior
Expected behaivour mentioned above.
Steps To Reproduce
No response
Environment
- OS: Amazon Linux 2
- Python: 3.9.5
- dbt: 1.7.0
- dbt-athena-community: 1.7.0
Additional Context
No response