Add support for column comments in config transformations
Closed this issue · 0 comments
yruslan commented
Background
Currently, you can define simple transformations as part of configuration like this:
transformations = [
{ col = "col1", expr = "lower(col2)" }
],
Feature
Add support for [optional] column comments in config transformations.
Also adding a comment can be the only operation of a transformation.
Example
transformations = [
{ col = "col1", comment = "A comment for existing col1" }
{ col = "col2", expr = "lower(col1)", comment = "A comment for new col2" }
],
This is useful if the table is exposed in Data Lake, Glue, or some other metastore.