akka/akka-persistence-dynamodb

Support for querying, updating etc the store using Akka Streams

Closed this issue · 0 comments

akka-persistence-dynamodb is very nice. But it's a bit cumbersome that you can't easily query ad hoc what events are in your journal. Or do a data migration because you missed something in your serialization logic. Etc.
So, in my experience, it's very nice to have some Scala functions that make it easy to implement your own Akka Streams queries on the dynamodb store.
I've implemented some of those for my project. And I'm willing to provide a PR implementing this issue.

I think implementing this issue raises the question of how to deal with the existing code and code duplication.
Currently most logic in this plugin is implemented using Futures and (non tail-) recursive functions. And mixins that pre suppose that the calling code is an Actor.

I've got this feeling that it would be nice to refactor that logic to use Akka Streams instead of Futures and recursive functions. Because that allows for more powerful logic wrt to dealing with AWS throttling during recovery. And it wouldn't use the stack.
And to move away from mixins that assume Actor. But use functions that expect an implicit ActorSystem, materializer, dynamodbclient.. Because cake pattern traits are quite cumbersome to reuse. (I find it really takes quite a bit of time finding out how to refactor the traits before they can be reused. It basically hinders development I find.)

Just to underscore the need for reuse of logic: the way that the plugin shards events of a single persistenceid and the lookup of highest sequence number of a single persistenceid is very performant. But it also makes querying prohibitive to implement oneself.

@johanandren @ktoso what do you think?