doublecloud/transfer

Alter add unknow default for column failed to applied to clickhouse

Opened this issue · 1 comments

When postgres source got more then 1 columns to add replication generate an error:

error failed to execute pg activate hook: 
Snapshot loading failed: unable to upload tables: 
upload of 9 tables failed: 
unable to finish table loading: 
unable to sent 'done_load_table' for table '"public"."companies" [1/1] (ETARows: 8)': unable push batch with a non row item: 
failed to push items from 0 to 8 in batch: 
failed to push (with retries): Push failed: 
failed to push 8 rows to ClickHouse shard 0: 
ClickHouse Push failed (got fatal error): unable to get table 'public_companies': 
failed to apply schema diff to public_companies: 
error executing DDL: 
code: 62, message: Syntax error: failed at position 200 (']'): ], 'Nullable(String)'), ADD COLUMN IF NOT EXISTS `default_data_language` Nullable(String), ADD COLUMN IF NOT EXISTS `additional_data_languages` Nullable(String). Expected one of: token, Comma

Here is a DDL:

ALTER TABLE `public_companies`  ON CLUSTER `default` 
ADD COLUMN IF NOT EXISTS `default_language` Nullable(String), 
ADD COLUMN IF NOT EXISTS `additional_languages` Nullable(String) 
     DEFAULT CAST(ARRAY[], 'Nullable(String)'), 
ADD COLUMN IF NOT EXISTS `default_data_language` Nullable(String), 
ADD COLUMN IF NOT EXISTS `additional_data_languages` Nullable(String) 
     DEFAULT CAST(ARRAY[], 'Nullable(String)')

Postgres source schema:

Column           |            Type             | Collation | Nullable |                  Default                  
---------------------------+-----------------------------+-----------+----------+-------------------------------------------
 company_id                | text                        |           | not null | 
 external_id               | text                        |           | not null | 
 title                     | text                        | en_US     | not null | 
 fullname                  | text                        | en_US     |          | 
 user_id                   | text                        |           |          | 
 updated                   | timestamp(0) with time zone |           | not null | now()
 created                   | timestamp(0) with time zone |           | not null | now()
 lsn                       | bigint                      |           | not null | nextval('companies_lsn_seq'::regclass)
 serial                    | bigint                      |           | not null | nextval('companies_serial_seq'::regclass)
 secret                    | text                        |           |          | 
 products_scope            | text[]                      |           | not null | ARRAY[]::text[]
 ownership                 | text                        |           | not null | 'yandex'::text
 instance_erp              | text                        |           |          | 
 users_manage              | text                        |           |          | 
 vars                      | jsonb                       |           | not null | '{}'::jsonb
 source                    | text                        |           |          | 
 country_code              | text                        |           | not null | 
 type                      | text                        |           | not null | 'original'::text
 default_language          | text                        |           |          | 
 additional_languages      | text[]                      |           |          | ARRAY[]::text[]
 default_data_language     | text                        |           |          | 
 additional_data_languages | text[]                      |           |          | ARRAY[]::text[]