[ADAP-454] [Bug] `api.Column.string_type` causes error
rlh1994 opened this issue · 5 comments
Is this a new bug in dbt-spark?
- I believe this is a new bug in dbt-spark
- I have searched the existing issues, and I could not find an existing issue for this bug
Current Behavior
Because there is no overwritten function, it uses a the core class, which is character varying
which doesn't exist in spark (or databricks), so this api call just errors in spark/databricks targets.
Expected Behavior
Model to pass, likely using string
as the type.
Steps To Reproduce
The easiest way to see it is with a basic model that just casts something to the string type:
{{
config(
materialized = 'table',
)
}}
select
cast(10 as {{api.Column.string_type(4000)}}) as test_col
Which in the case of a databricks target leads to the following compiled code
select
cast(10 as character varying(4000)) as test_col
and a dbt run
gives the error output
dbt run --target databricks
10:05:06 Running with dbt=1.4.5
10:05:06 Unable to do partial parsing because config vars, config profile, or config target have changed
10:05:06 Unable to do partial parsing because a project dependency has been added
10:05:07 Found 1 model, 0 tests, 0 snapshots, 0 analyses, 374 macros, 0 operations, 0 seed files, 0 sources, 0 exposures, 0 metrics
10:05:07
10:08:17 Concurrency: 1 threads (target='databricks')
10:08:17
10:08:17 1 of 1 START sql table model dbt_ryan.test_models .............................. [RUN]
10:08:17 1 of 1 ERROR creating sql table model dbt_ryan.test_models ..................... [ERROR in 0.64s]
10:08:18
10:08:18 Finished running 1 table model in 0 hours 3 minutes and 10.96 seconds (190.96s).
10:08:18
10:08:18 Completed with 1 error and 0 warnings:
10:08:18
10:08:18 Runtime Error in model test_models (models/test_models.sql)
10:08:18
10:08:18 [PARSE_SYNTAX_ERROR] Syntax error at or near 'varying': missing ')'.(line 19, pos 25)
10:08:18
10:08:18 == SQL ==
10:08:18 /* {"app": "dbt", "dbt_version": "1.4.5", "dbt_databricks_version": "1.4.2", "databricks_sql_connector_version": "2.3.0", "profile_name": "default", "target_name": "databricks", "node_id": "model.dbt_demo.test_models"} */
10:08:18
10:08:18
10:08:18
10:08:18 create or replace table `dbt_ryan`.`test_models`
10:08:18
10:08:18
10:08:18 using delta
10:08:18
10:08:18
10:08:18
10:08:18
10:08:18
10:08:18
10:08:18 as
10:08:18
10:08:18
10:08:18 select
10:08:18 cast(10 as character varying(4000)) as test_col
10:08:18 -------------------------^^^
10:08:18
10:08:18
10:08:18
Relevant log output
No response
Environment
- OS: MacOS Ventura 13.3.1 (22E261)
- Python: 3.9.13
- dbt-core: 1.4.5
- dbt-spark: 1.4.1
Additional Context
Issues in BQ: dbt-labs/dbt-bigquery#665
dbt-core relevant issue: dbt-labs/dbt-adapters#81
This issue has been marked as Stale because it has been open for 180 days with no activity. If you would like the issue to remain open, please comment on the issue or else it will be closed in 7 days.
This issue has been marked as Stale because it has been open for 180 days with no activity. If you would like the issue to remain open, please comment on the issue or else it will be closed in 7 days.