Magnolify/tensorflow doesn't support optionals and nested repeated fields

[comeonism]

Currently the docs here state that "Optional and repeated types are not supported in a nested field", is there a plan to add a fix/workaround that would make using those possible?

Currently optionals and nested repeated fields are the main thing standing between us and an upgrade to Scio 10.

Here's a rough example of what our case classes look like:

  case class CaseClass1(
    id: String,
    number: Option[Long],
    someFeatures: Option[SomeFeaturesCaseClass]
  )

  case class CaseClass2(
    baseFeatures: CaseClass1,
    otherFeatures: Map[String, OtherFeaturesCaseClass],
    yetOtherFeatures: Map[String, YetOtherFeaturesCaseClass]
  )

  val CASE_CLASS_CONVERTER: ExampleType[CaseClass2] =
    ExampleType[CaseClass2]

[nevillelyh]

No because TFE has a single list per field while repeated in optional is technically a list of a list so an empty list could be interpreted as Some(Nil) or None, among other potential regrets. We prefer these being hacked into user space at their own discretion.

[comeonism]

Can you provide any example code snippets on how we might do that?