Reduce the Requirements on User Defined Types

[keithwill]

VestPocket allows the consumer of the library to store and retrieve objects of their own types, but this requires that those types inherit from a type that implements IEntity, as that interface contains the key, version, and deleted status. A change to an entity is stored by serializing the object to disk (using System.Text.Json).

As the object has to implement IEntity, it guarantees that the serialized entity also doubles as a VestPocket 'record' of the state of that entity. Requiring inheritance from a class that implements IEntity makes it possible to use polymorphic System.Text.Json, so that entities can be eagerly loaded from JSON to objects at startup (through the use of the $type property that STJ will include for polymorphism support).

This was a convenient approach when developing VestPocket, but is a subpar developer experience for consumers of the library. While the base type is defined within the library consumer's project (and they can control it as desired), it is more typical for POCOs and model classes to not require inheritance, or to allow more natural inheritance based on domain concerns (instead of storage concerns).

The use of polymorphism also means that we cannot take advantage of fast-path serialization even though we are utilizing source generation, see STJ polymorphism notes.

The purpose of this issue is the removal of user defined types having to implement IEntity, and the removal of user defined types having to inherit from a base class owned by the library consumer. A partial success would be the removal of IEntity, but still requiring inheritance from either a user defined base type or a base type defined within VestPocket.

The Removal of IEntity

The fields in IEntity need to be stored separately by VestPocket instead of being serialized as part of the entity. The main approach will be to serialize a JSON object that wraps the entity; a 'record'. This record will be a key value pair. The object currently serialized as a record will instead become the value in the key value pair. A record might look like the following:

{"key":"person/john-doe","val":{"$type":"Person","name":"John Doe"}}

Separating the JSON of an entity from the record that wraps it in a field also allows a convenient way to express that a value has been deleted or removed:

{"key":"person/john-doe","val":null}

See #6 for an explanation of why 'Version' is being removed.

The Removal of Polymorphic Inheritance

Polymorphic inheritance provides several benefits to VestPocket. It ensures that the user of the library is setting up a class hierarchy where every type involved implements IEntity, and it enables polymorphic serialization using System.Text.Json.

With the removal of IEntity, the first concern will no longer be useful, but the concern of polymorphic serialization still needs to be dealt with.

There are several potential approaches:

A JSON property for $type could be added to the 'record' (the object that wraps the serialized entity). It will have to be investigated how to map the type names to generic types so that a strongly typed JsonTypeInfo can be retrieved when a VestPocketStore is being initialized and records are being loaded eagerly.

System.Text.Json already does this on its own for polymorphic serialization. VestPocket currently only requires the JsonTypeInfo for the 'base' type that implements IEntity that all of the library consumer's types inherit from. With this approach we may need access to the consumer's JsonSerializerContext instead of a single JsonTypeInfo. It should contain some meta data we can use to translate the $type parameter into a Type object that we can use with the JsonSerializer overloads that take a JsonSerializerContext and a Type object.

A concern with this approach is that STJ will expect the $type parameter to exist on the entity and can't find one on our wrapping entity. If the library consumer wants to utilize polymorphic serialization after we no longer require it in VestPocket, we'll have to ensure that whatever mechanism we use to generate our own $type property for the record object does not conflict with one that might be added when serializing the user defined type of the entity.

For example, we might create a record like this in JSON:
{"key":"person/john-doe","$type":"Person","val":{"name":"John Doe"}}

...but what if the Person type is a base class in a hierarchy, and the library consumer has configured it for polymorphic serialization and a subclass of 'Customer'. When they serialize a customer object, what would our resulting record JSON look like?

It could end up looking like this:
{"key":"person/john-doe","$type":"Customer","val":{"$type":"Customer","name":"John Doe"}}

When making the call serialize or deserialize this object, it will have to be investigated if a JsonSerializer call can handle getting a subclass Type directly instead of being passed the type of the base class (I strongly suspect it can). VestPocket will have to understand that a hierarchy exists in the user defined type stored, based only on the outer type name supplied, and know which type is the best to supply when deserializing.

Move to loading entities lazily and relying on the caller of a get/search operation to supply the type before the entity is deserialized. With this approach, the 'records' would be parsed at startup, but entities would not be deserialized into objects. Instead, the offsets into the file would be recorded (or an array of the byte data of its json) and an object would be lazily deserialized the first time that a consumer of the library accesses the associated key. This approach would raise additional questions, such as what when the consumer of the library supplies the wrong type for a key lookup or search range; there wouldn't be a way to understand this had happened without storing type information about each key.

Continue to require inheritance. This isn't preferable, as library consumers won't enjoy inheriting, and we'll lose fast-path source generated serialization, but it could be combined with other mitigations. With the removal of IEntity and other planned changes to versioning, there aren't any state or behaviors required to be part of an inheritable marker class used for polymorphism.

We could add an Entity 'record class' to VestPocket and require users of the library to inherit from it. This would be the only way to require all types stored in VestPocket to be record class types. There is currently no other way to create a generic type constraint ensuring that record classes are used. This would ensure VestPocket library users are guided to the proper experience for VestPocket; assuming they stick to adding properties as auto properties on the constructor, they would end up using the only type of language supported immutable types.