can't access colum when TypedDataset of primitive types
ayoub-benali opened this issue · 3 comments
It seems like frameless isn't aware of the TypedDataset schema when it contains only primitive types. Simple example to generate one would be:
val ds1: TypedDataset[Int] = TypedDataset.create(List(1, 2, 3))
val col = ds1.col('_1)
Leads to
[E] No column Symbol with shapeless.tag.Tagged[String("_1")] of type A in Int
[E] L28: ds2.col('_1)
Eventhough printSchema
shows:
root
|-- _1: integer (nullable = false)
Adding an .as[Tuple1[Int]]
creates an other error:
could not find implicit value for parameter as: frameless.ops.As[Int,(Int,)]
After some operations on TypedDataset one can easily end-up with dataset of primitive types and be blocked.
I feel there should be at least some implicits in framless to provide an As
for primitive types
frameless: 0.8.0
I see that @kanterov recommended helping with an implicit As
here: https://gitter.im/typelevel/frameless/archives/2017/03/04?at=58baff3a21d548df2c8e2781
But this isn't possible with current API. I don't know if it is regression.
implicit val intAsTuple1: As[Int, Tuple1[Int]] = As.deriveAs[Int, Tuple1[Int]]
generates this error "could not find implicit value for parameter i1: frameless.ops.As.Equiv[Int,(Int,)]"
@ayoub-benali absolutely. Let me close this one.