Composite sort key ergonomics

[jhenahan]

AWS Docs: https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/bp-sort-keys.html

Context:

Suppose the following type represents an entry in a Dynamo table:

case class Entry(
    partitionKey: String,
    compositeSortKey: Rich,
    date: LocalDate
)

object Entry {
  implicit val entryDynosaurSchema: DynosaurSchema[Entry] =
    DynosaurSchema.record { field =>
      (
        field("partitionKey", _.partitionKey)(DynosaurSchema[String]),
        field("compositeSortKey", _.compositeSortKey)(DynosaurSchema[Rich]),
        field("date", _.date)(
          DynosaurSchema[LocalDate]
        )
      ).mapN(Entry.apply)
    }

  implicit val entryMeteorCodec: MeteorCodec[Entry] =
    schemaToCodec[Entry](entryDynosaurSchema)
}

We define a composite sort key by:

case class Rich(field1: String, field2: LocalDate)

object Rich {
  private def parseField1(
      field1: String
  ): Either[ReadError, String] = ...

  private def parseField2(
      field2: String
  ): Either[ReadError, LocalDate] = ...

  implicit val richSchema: DynosaurSchema[Rich] =
    DynosaurSchema[String].imapErr {
      case s"$field1#$field2" =>
        (parseField1(field1), parseField2(field2)).mapN(
          Rich.apply
        )
      case v => ReadError(s"Bad composite: $v").asLeft
    }(r => show"${r.field1}#${r.field2.toEpochDay}")

  implicit val richMeteorCodec: MeteorCodec[Rich] = schemaToCodec(
    richSchema
  )
}

(It is hinted at above, but for the sake of clarity I'll note that I've written machinery to normalize LocalDate to a Long for comfortable reading and writing. It's not relevant to the issue at hand, but I want the behavior described to make sense)

So, when I go to write entries, I can write an Entry("someId", Rich("low-cardinality", LocalDate.now()), LocalDate.now()) and get a row in Dynamo like

partitionKey	sortKey	date
someId	low-cardinality#12345	12345

This is all lovely, but I'd like to be able to use a SortKeyQuery to target my queries more specifically. In particular, I'd like to be able to say BeginsWith("low-cardinality"), but I can't since the S has to be Rich. E.g.,

val byCompositeKey: CompositeTable[IO, String, Rich] =
  CompositeTable(
    "example-table",
    KeyDef[String]("partitionKey", DynamoDbType.S),
    KeyDef[Rich]("sortKey", DynamoDbType.S),
    ddb
  )

val byCompositeKeyUnsafe: CompositeTable[IO, String, String] =
  CompositeTable(
    "example-table",
    KeyDef[String]("partitionKey", DynamoDbType.S),
    KeyDef[String]("sortKey", DynamoDbType.S),
    ddb
  )

The first only allows me to construct SortKeyQuery[Rich], while the second lets me do whatever with the string rep at the cost of type safety. I'd really love to be able to represent a more principled query, but I don't know if the tools exist in the library to do so.

This feels like something that could be solvable with optics, but I'm not quite sure how to go about it. It seems like I ought to be able to construct a Prism[Rich, String] (the real code uses newtypes pervasively, so it wouldn't actually be String) and then use that to construct a SortKeyQuery specialized to one or more parts of the sort key.

I'm on vacation for the next couple of weeks, so I may see if I can grok enough of the code that's already here to prove out that idea, but I figured I'd throw the idea out in case it gave you an AHA! moment.

[mcanlas]

I had also wondered what the type safe version of this was, but took a different approach.

Basically for every "rich" composite sort key (A, B, C) you should be able to query starts with using (A) and (A, B) as well

sealed trait CompositeValue2[A, +B]

final case class One[A](x: A) extends CompositeValue2[A, Nothing]

object CompositeValue2 {
  implicit def attrKey2Encoder[A, B](implicit
    A: DynamoStringEncoder[A]
  ): Encoder[CompositeValue2[A, B]] =
    Encoder[String]
      .contramap { case One(x) =>
        A.toDynamoString(x)
      }
}

So every composite value One(a) or Two(a, b) would extend Three[A, B, C] and use Nothing/inheritance.