Collection flows with incremental events

[rnett]

Use case

I have encountered several scenarios where I want to have a "live" collection. For example, a map of ids to connections, where incoming connection requests cause a connection to be added. I want to be able to react to updates to the map - for example, a server that starts listening to each new connection.

This is kind of possible with the current api by using something like a Flow<Map<Id, Connection>>, however it falls short in a couple of important ways:

1. The whole map is re-created and updated each time. It's a waste of memory, and it's easy to cause concurrency issues unless you use MutableStateFlow.update. This can be somewhat mitigated by using PersistentMap.
2. It's impossible to have a long-lived collector for each key - the collector is canceled and re-executed for all keys for every change. For example, for the connection example, I want to start listening to the connection for each key once it's added to the map, and stop when it's removed.

The Shape of the API

The basic API isn't terribly complicated. For maps, it might look something like:

sealed interface MapEvent<out K, out V> {
  data class Put<out K, out V>(val k: K, val v: V): MapEvent<K, V>
  data class Remove<out K>(val k: K): MapEvent<K, V>
}

fun <K, V> Flow<Map<K, V>>.asEvents(): Flow<MapEvent<K, V>>

However, that doesn't handle the state management. Ideally there would also be something like:

val map = MapStateFlow<Id, Connection>() // extends Flow<MapEvent<K, V>> - may just wrap it and provide map-like accessors

There's also the other collection types to consider. There's threemain API design channels that I can see:

• Generalizing vs specializing for each collection type
• Specialized flows for these types, aka how to provide a collection-like interface that isn't just a flow of events
• Batch events (i.e. for putAll) for efficiency

Prior Art

None that I'm aware of. Compose's SnapshotState collections come closest, and also demonstrate the general need for "live" collection types. They don't suffer from the "collection restarted every change" issue because while re-composition happens every change, it's smart enough to re-use re-occurring keys. But there might be some ways to optimize it with a construct like this.

[dkhalanskyjb]

Duplicate of #3316