dst table was truncated during transfer migration
Closed this issue · 1 comments
rifler commented
I was migrating SNAPSHOT_AND_INCREMENT
pg -> ch transfer from DoubleCloud to k8s with id preservation.
spec
transfer_spec:
id: dttlkh8khl5e34ee9g3p
type: SNAPSHOT_AND_INCREMENT
src:
type: pg
params:
Hosts:
- ${DATATRANSFER_SRC_HOSTNAME}
Port: 5432
User: ${DATATRANSFER_SRC_USERNAME}
Password: ${DATATRANSFER_SRC_PASSWORD}
Database: xxxx
EnableTLS: true
dst:
type: ch
params:
ShardsList:
- Hosts:
- ${DATATRANSFER_DST_HOSTNAME}
User: ${DATATRANSFER_DST_USERNAME}
Password: ${DATATRANSFER_DST_PASSWORD}
Database: yyyy
SSLEnabled: true
HTTPPort: 8443
NativePort: 9440
Cleanup: Truncate
Interval: 1000000000
RetryCount: 20
UseSchemaInTableName: true
InsertParams:
MaterializedViewsIgnoreErrors: true
data_objects:
include_objects:
- public.history
Replication slot in PG was preserved and reused, but initial snapshot was also done, reread ~2M rows.
Logs from snapshot pod:
run activate with: *s3coordinator.CoordinatorS3
coordinator/coordinator_fake_client.go:96 fake change status: dttlkh8khl5e34ee9g3p -> Started
slot dttlkh8khl5e34ee9g3p in database xxxx exist: true
replication slot already exists, try to drop it
slot dttlkh8khl5e34ee9g3p in database xxxx exist: true
Will try to delete slot
Drop slot query executed
Slot should be deleted, double check
slot dttlkh8khl5e34ee9g3p in database support_chat exist: false
a replication slot already exists, Will sleep 677.78146ms and then retry create replication slot because of an error.
slot dttlkh8khl5e34ee9g3p in database xxxx exist: true
Will try to delete slot
Drop slot query executed
Slot should be deleted, double check
slot dttlkh8khl5e34ee9g3p in database support_chat exist: false
a replication slot already exists, Will sleep 677.78146ms and then retry create replication slot because of an error.
slot dttlkh8khl5e34ee9g3p in database xxxx exist: false
Create slot
Replication slot created, re-check existence
slot dttlkh8khl5e34ee9g3p in database xxxx exist: true
need to cleanup (Truncate) tables
rifler commented
Incremental pod (from sts) also contains logs about recreating replication slot and truncating table