citusdata/citus

Documentation request: Citus cluster migration

Opened this issue · 0 comments

TLDR - I'd like to have documentation covering Citus cluster migration scenarios

Our background (the non-TLDR part) is covered in Citus Slack

To successfully migrate a complete Citus cluster the needed steps are somewhat clear:

  • deploy a new cluster
  • copy the data over and keep it in sync
  • switch over the clients to the new cluster

I believe it's possible to replicate a (current) Citus cluster into empty new Citus cluster (with same number of workers) using PUBLICATION + SUBSCRIPTION across each old and new workers, and across the old and new coordinators. By reading the documentation about CDC the idea is clear - just create pubs+subs not for ALL TABLES, but for the non-metadata user tables. But as we're replicating each worker and coordinators separately it's unclear if citus.enable_change_data_capture brings anything on the table (as we want to each target worker / shard to get 1:1 changes from the source)

It would be awesome if this kind of "migrate your Citus into new hardware" type of scenario would be covered in the documentation.