This repo contains two sub-projects related to schema migration: REST Exporter and Connect Transform. Since they each work in different ways, they offer unique advantages and disadvantages: use whichever suits your circumstances.
This Python project provides a batch import/export capability of subjects. There are two tools available:
- exporter: this reads schemas from a Schema Registry REST endpoint and writes them to a local file
- importer: this reads schemas from a local file and writes them to a
_schemastopic on a target Redpanda cluster
This Java project contains a Kafka Connect Single Message Transform (SMT) that can enables the use of MirrorMaker 2 for continuous, real-time migration of schemas between clusters.
-
If you only have visibility of the REST endpoint for Schema Registry, and not the underlying
_schemastopic, the REST exporter is the only available choice. -
If you have visibility of the
_schemastopic on the source cluster, then either the REST exporter or MirrorMaker 2 (with the transform) can be considered.
The use of MirrorMaker 2 means a more complex deployment, but the advantages of being both realtime and continuous are obvious.
Both approaches provide a unidirectional replication capability. If the target cluster is usable, there is a risk that new subjects / schemas are created, resulting in a schema ID clash.
This should be avoided while replication is ongoing.