typelevel/frameless

can't access colum when TypedDataset of primitive types

ayoub-benali opened this issue · 3 comments

It seems like frameless isn't aware of the TypedDataset schema when it contains only primitive types. Simple example to generate one would be:

    val ds1: TypedDataset[Int] = TypedDataset.create(List(1, 2, 3))
    val col = ds1.col('_1)

Leads to

[E]      No column Symbol with shapeless.tag.Tagged[String("_1")] of type A in Int
[E]      L28:     ds2.col('_1)

Eventhough printSchema shows:

root
 |-- _1: integer (nullable = false)

Adding an .as[Tuple1[Int]] creates an other error:
could not find implicit value for parameter as: frameless.ops.As[Int,(Int,)]

After some operations on TypedDataset one can easily end-up with dataset of primitive types and be blocked.

I feel there should be at least some implicits in framless to provide an As for primitive types

frameless: 0.8.0

I see that @kanterov recommended helping with an implicit As here: https://gitter.im/typelevel/frameless/archives/2017/03/04?at=58baff3a21d548df2c8e2781
But this isn't possible with current API. I don't know if it is regression.

implicit val intAsTuple1: As[Int, Tuple1[Int]] = As.deriveAs[Int, Tuple1[Int]]

generates this error "could not find implicit value for parameter i1: frameless.ops.As.Equiv[Int,(Int,)]"

@imarios should we consider this issue fixed by #475 ?

@ayoub-benali absolutely. Let me close this one.