Iterate over TypedColumns with evidence
gdahia opened this issue · 0 comments
gdahia commented
I was wondering if there was a way to iterate over the columns as Symbol
s with the appropriate evidence of their existence in the typed dataset, and perhaps some type tagging to ensure the operations for that symbol are type compatible.
My concrete example is the following: I have a dataset for which some columns are String
, and want to replace those columns by translated version of their contents. Some (pseudo)code illustrating what I wanted to achieve is
val ds = TypedDataset(...)
ds.columns.foldLeft(ds) { (ds, typedColumn) =>
typedColumn match {
case typedColumn: TypedColumn[String] => ds.withColumnReplaced(typedColumn.symbol, translate(typedColumn))
case _ => ds
}
}
Is that already possible?
Thanks!