[Platform] xCluster replication management through Platform
ymahajan opened this issue · 1 comments
ymahajan commented
Motivation
- Easy xCluster replication setup: xCluster replication is currently a supported feature in Yugabyte DB. However, the only way to set it up is through a yb-admin command (CLI) which can be complicated and error prone to use.
- Ensure setup correctness: Currently, there is no easy way to find out if the replication is enabled or not for a universe or there is no way to track relationships between source and target universe.
- Monitor and track xCluster replication: There is no easy way to understand and monitor the overall configuration as well as the current state of replication.
Our goal is to manage the xCluster replication setup, configuration and monitoring through YB Platform.
Phase 1
| Status | Subtask | GitHub Issue | Estimated Time |
|---|---|---|---|
| ✅ | Add tracking of xCluster Replication Status to universe | #8769 | |
| ✅ | Add an API for syncing platform with cluster state | #8917 | |
| ✅ | Modify get universe API to contain replication information | #8918 | 2 days |
| 🛠 | Update replication map when cluster master addresses are changed | #9390 | 3 days |
| ✅ | Create library methods in yb-client to support xCluster Async Replication | #9123 | 7 days |
| ✅ | Define Tasks for AsyncReplicationRelationship management | #9122 | 7 days |
| ✅ | Create REST endpoints + controller for managing xCluster async replication relationships | #9124 | 3 days |
| ⬜️ | Check for async replication relationships during software updates | #9125 | 3 days |
| ⬜️ | Define new metrics for async replication lag on the table level | #9126 | 6 days |
| ⬜️ | [Stretch Goal] Check for schema changes for tables with async replication enabled | #9127 | |
| ⬜️ | UI support for xCluster replication management phase-1 | #9566 |
Phase 2
| Status | Subtask | GitHub Issue |
|---|---|---|
| ⬜️ | Apply schema changes through Platform | |
| ⬜️ | Automatically bootstrapping target universes | |
| ⬜️ | Allow safe DDL changes to be propagated automatically to target universes | |
| ⬜️ | Initialize replication: When adding a new table to be replicated, it should seed the target universe automatically. | |
| ⬜️ | Provide time estimate to init replication | |
| ⬜️ | Ability to manage replication between universes managed by two different platforms | |
| ⬜️ | Universe or schema configured for replication should automatically create or drop objects in the target universe |
karishmalalani commented
Following are some suggestions [Replication TAB]that to easily identify any Table or Row Count discrepancies between Producer & Consumer:
-
Table structure is different between both sides
-
Row counts are off between both sides - at the Table level
-
Table existing on one side and not the other
-
Provide a way to retrieve a summary/report of Replicated Tables and any issues between the 2 sides at table level