Add Schema Registry Documentation to Snowflake Column Comments
Closed this issue · 1 comments
wornjs commented
When creating tables using the Snowflake Kafka Connector, the column comment is always set to "created by schema evolution from Snowflake Kafka Connector". If there is information available in the Schema Registry's connect.doc, we should use that value as the column comment instead.
Example:
When using Debezium to capture CDC (Change Data Capture) data, column comments are stored in the Schema Registry. By utilizing these comments, we can ensure that the column comments in Snowflake match the original database comments exactly. This would improve the consistency and readability of the data schema.
Benefits:
- Improved consistency between source and target schemas
- Enhanced readability and understanding of column purposes
- More informative column comments in Snowflake tables
Proposed Change:
- Modify the Snowflake Kafka Connector to check for connect.doc values in the Schema Registry.
- If a connect.doc value exists, use it as the column comment.
- If no connect.doc value is available, default to the current comment "created by schema evolution from Snowflake Kafka Connector".
sfc-gh-akowalczyk commented