Multi-connector connector
Closed this issue · 0 comments
tmnd1991 commented
Multi-Connector
Multi-connector can connect to multiple connectors in a hierarchical order. It is useful when schemas are registered on different datastore (i.e. confluent + hbase).
You configure it in the following way:
darwin {
type = "lazy"
connector = "multi"
registrator = "hbase"
confluent-single-object-encoding: ["confluent"]
standard-single-object-encoding: ["hbase", "mongo"]
confluent {
endpoints: ["http://schema-registry-00:7777", "http://schema-registry-01:7777"]
max-cached-schemas: 1000
}
hbase {
isSecure: false
namespace: "DARWIN"
table: "REPOSITORY"
coreSite: "/etc/hadoop/conf/core-site.xml"
hbaseSite: "/etc/hadoop/conf/hbase-site.xml"
}
mongo {
username = "mongo"
password = "mongo"
host = ["localhost:12345"]
database = "test"
collection = "collection_test"
timeout = 5000
}
}
When extracting the schemaId, it will check if the single object encoding is "confluent" or "standard" way. Given that, it will go down the chain of confluent-single-object-encoding or standard-single-object-encoding in order. The first that matches, is the one that will be used.
Register of schema, will work with the connector set as registrator.