Incremental materialization strategy delete+insert fails when one of the previous records was updated
Daniel-Itzul opened this issue · 0 comments
Describe the bug
A clear and concise description of what the bug is. What command did you run? What happened?
dbt run
After running an incremental model (delete+insert strategy) on a delta on original data, where some data points where updated.
The following error is shown
Database Error in model all_orders (models\core\all_orders.sql)
13:25:49 [Version 17.20.0.19] [Session 2702] [Teradata Database] [Error 3707] Syntax error, expected something like '(' between the 'column' keyword and
the word 'order_status__dbt_alter'.
Notice that the table dbt_alter is a dbt-teradata connector generated table not a table generated through user defined models.
Steps To Reproduce
In as much detail as possible, please provide steps to reproduce the issue. Sample data that triggers the issue, example model code, etc is all very helpful here.
-
Clone this repository: https://github.com/Teradata/teddy_retailers_dbt-dev/
-
Create the teddy_retailers database in your vantage instance (created through ClearScape Analytics Experience due to NOS compatibility)
-
Run the following script in your vantage instance
CREATE SET TABLE teddy_retailers.source_products AS ( SELECT * FROM ( LOCATION='/gs/storage.googleapis.com/clearscape_analytics_demo_data/DEMO_dbtAdvanced/raw_products.csv') as products ) WITH DATA; CREATE SET TABLE teddy_retailers.source_customers AS ( SELECT * FROM ( LOCATION='/gs/storage.googleapis.com/clearscape_analytics_demo_data/DEMO_dbtAdvanced/raw_customers.csv') as customers ) WITH DATA; CREATE SET TABLE teddy_retailers.source_orders AS ( SELECT * FROM ( LOCATION='/gs/storage.googleapis.com/clearscape_analytics_demo_data/DEMO_dbtAdvanced/primary_orders_first.csv') as orders ) WITH DATA; CREATE SET TABLE teddy_retailers.source_order_products AS ( SELECT order_id, product_id, product_quantity FROM ( LOCATION='/gs/storage.googleapis.com/clearscape_analytics_demo_data/DEMO_dbtAdvanced/primary_order_products_first.csv') as orders_products ) WITH DATA;
-
cd to the cloned repository and execute dbt_run
-
Update the data sources for the models with the following script
drop table teddy_retailers.source_orders; CREATE SET TABLE teddy_retailers.source_orders AS ( SELECT * FROM ( LOCATION='/gs/storage.googleapis.com/clearscape_analytics_demo_data/DEMO_dbtAdvanced/primary_orders_second.csv') as orders ) WITH DATA; drop table teddy_retailers.source_order_products; CREATE SET TABLE teddy_retailers.source_order_products AS ( SELECT order_id, product_id, product_quantity FROM ( LOCATION='/gs/storage.googleapis.com/clearscape_analytics_demo_data/DEMO_dbtAdvanced/primary_order_products_second.csv') as orders_products ) WITH DATA;
-
Execute dbt run once again.
Expected behavior
The incremental model should run correctly without erroring due to syntax errors in internal generated commands.
Screenshots and log output
`alter table "teddy_retailers"."all_orders" add column "order_status__dbt_alter" character varying(17);
update "teddy_retailers"."all_orders" set "order_status__dbt_alter" = "order_status";
alter table "teddy_retailers"."all_orders" drop column "order_status" cascade;
alter table "teddy_retailers"."all_orders" rename column "order_status__dbt_alter" to "order_status"
�[0m13:19:22.240566 [debug] [Thread-1 (]: teradata adapter: Teradata error: [Version 17.20.0.19] [Session 2678] [Teradata Database] [Error 3707] Syntax error, expected something like '(' between the 'column' keyword and the word 'order_status__dbt_alter'.
at gosqldriver/teradatasql.formatError ErrorUtil.go:89`
The output of dbt --version
:
1.3.3
The operating system you're using:
Windows 10
The output of python --version
:
3.10
Additional context
Add any other context about the problem here.