Teradata/dbt-teradata

Incremental Materization : Merge not working when update does single column

clydewatts1 opened this issue · 0 comments

Describe the bug

Using A Incremental Materilisation of a single update column cause DBT to abend.


-- This is a simple BK Process
-- Surrogate Key : Serial Monotonic
-- Comments: This could be any table.


{{
config(
materialized='incremental',
schema='T_SKEY',
unique_key='EDW_KEY',
on_schema_change='fail',
incremental_strategy='merge',
index="PRIMARY INDEX ( SOURCE_KEY )"
)
}}
-- Get all the Natural Keys
WITH S AS (
SELECT NBO.ORG_NUM||''||NBO.ORG_TYPE_CODE||''||TRIM(NBO.SYSTEM_RECORD_INDICATOR) (VARCHAR(12)) AS SOURCE_KEY
FROM {{source('SRCI_RMS_NB','NB_W_INT_ORG_DS')}} AS NBO
WHERE RECORD_DEL_IND=0
AND END_DT=DATE '3500-12-31'
)

{% if is_incremental() %}
-- If it is an incremental run then get the last id
,SMAX AS (
SELECT MAX(EDW_KEY) AS MAX_IDW_KEY
FROM {{this}}

)
,FINAL AS (
SELECT SOURCE_KEY
,ROW_NUMBER() OVER (ORDER BY SOURCE_KEY) + (SELECT MAX_IDW_KEY FROM SMAX) (BIGINT) AS EDW_KEY
FROM S
WHERE NOT EXISTS (SELECT 1 FROM {{this}} AS X WHERE X.SOURCE_KEY = S.SOURCE_KEY)
)
{% else %}
-- if this is first time then "this" does not exist
,FINAL AS (
SELECT SOURCE_KEY
,ROW_NUMBER() OVER (ORDER BY SOURCE_KEY) (BIGINT) AS EDW_KEY
FROM S
)
{% endif %}
SELECT
SOURCE_KEY
,EDW_KEY

FROM FINAL

Steps To Reproduce

Created a model as above in models/bk_simple_006.sql , and ran initial , and incremental run.

dbt -d run --select bk_simple_006

Expected behavior

The merge statement is not supposed to abend/error out

Screenshots and log output

1:05:59 Using teradata connection "model.primark_poc.bk_simple_006"
11:05:59 On model.primark_poc.bk_simple_006: /* {"app": "dbt", "dbt_version": "1.6.7", "profile_name": "primark_poc", "target_name": "dev", "node_id": "model.primark_poc.bk_simple_006"} */

merge into "DWT10T_SKEY"."bk_simple_006" as DBT_INTERNAL_DEST
    using "DWT10T_SKEY"."bk_simple_006__dbt_tmp" as DBT_INTERNAL_SOURCE
    on
            DBT_INTERNAL_SOURCE.EDW_KEY = DBT_INTERNAL_DEST.EDW_KEY



when matched then update set


        **"SOURCE_KEY" = DBT_INTERNAL_SOURCE."SOURCE_KEY",**




when not matched then insert
    ("SOURCE_KEY", "EDW_KEY")
values
    (
        DBT_INTERNAL_SOURCE."SOURCE_KEY", DBT_INTERNAL_SOURCE."EDW_KEY"
    )

11:05:59 teradata adapter: Teradata error: [Version 20.0.0.0] [Session 6304053] [Teradata Database] [Error 3707] Syntax error, expected something like a name or a Unicode delimited identifier or a 'ROWID' keyword between ',' and the 'when' keyword.

The output of dbt --version:

<output goes here>

The operating system you're using:

WINDOWS 10 with latest release dbt-teradata

(dbt_primark) (base) C:\Users\cwatts\OneDrive - Primark\Documents\dbt_primark\primark_poc>pip show dbt-teradata
Name: dbt-teradata
Version: 1.6.7
Summary: The Teradata adapter plugin for dbt (data build tool)
Home-page: https://github.com/Teradata/dbt-teradata
Author: Teradata Corporation
Author-email: developers@teradata.com
License: UNKNOWN
Location: C:\Users\cwatts\OneDrive - Primark\Documents\dbt_primark\Lib\site-packages
Requires: dbt-core, teradatasql
Required-by:

(dbt_primark) (base) C:\Users\cwatts\OneDrive - Primark\Documents\dbt_primark\primark_poc>

Additional context

Add any other context about the problem here.