StarRocks/dbt-starrocks

StarRocks DBT connector to support schema evolution

Opened this issue · 0 comments

Does our StarRocks DBT connector handle the following DBT schema evolution changes? dbt-labs/dbt-spark#124 (comment)

Handle schema evolution with and without partitioning.

* The full import, without partitioning. As @charlottevdscheun mentioned earlier in https://github.com/dbt-labs/dbt-spark/pull/117. Currently we use INSERT INTO, but I would suggest to replace this by CREATE OR REPLACE TABLE, this allows us to atomically update the table, but also allowing changing the schema. This will keep full history of the table, and is fully supported by Delta.
* In the case of partition by, forward compatible schema evolution is allowed. We can add new fields to partitions, and they will be just null for the other partitions.