Will it work well with async/await ecosystem?

[TennyZhuang]

It seems that it’s unnecessary for a stackful coroutine implementation to mark a function as async, but async/await was standardized in rust and many libraries will build on top it. Is it compatible with lunatic?

[bkolobara]

Hi @TennyZhuang,

this question comes up from time to time, so I will explain here my thinking around this topic.

First of all, I would love to support async libraries in lunatic applications, but it's a bit of a complicated situation at the moment. Here are some of the current issues that make it very hard to support most async libraries in lunatic.

1. Async ecosystem split

There are not many truly "general purpose" async libraries. Most async libraries are heavily bound to an executor. This comes from the fact that async is heavily platform dependent and requires cooperation between the operating system, the library and the chosen executor. Even libraries that claim to be "runtime agnostic", like SQLx aren't really. They only have support for all the different runtimes because someone added it. There are bunch of other async executors out there that don't work with SQLx. This is even worse for the new kind of io_uring executors. I'm just randomly using SQLx as an example, but I really can't think of a rust async framework/library that works with any executor out there.

We could expose an executor in lunatic, but someone is going to need to go through all the libraries and explicitly add support for it. This can't happen automatically. So, if the libraries already need to be changed for lunatic, why not just add support directly for it? Instead of creating another level of indirection and creating an executor inside of lunatic.

It's true that the keywords async/await were standardised, but nothing else was. Not event AsyncRead/AsyncWrite (we don't have async traits yet, but this is yet another different issue), so every executor ships with its own TcpStream implementation and almost every library depends on a specific implementation of networking.

2. Different programming model

Most rust frameworks (including async) are built around a concurrency model based on threads. This kinda makes them a bad fit for lunatic. Lunatic requires you to design your application around small processes that don't share any memory. Even if we added support for them, it would feel a bit awkward.

Because of this, lunatic favours smaller libraries that do one thing well and work with standard Read/Write traits. It's just easier to incorporate them into this process paradigm. These libraries are also the only option in many other rust environments, like no_std. What I want to say here is that maybe limitations are good? Limiting lunatic to a set of well designed simple sync libraries, instead of the messy async world, could lead to great usability. I have written already enough async rust to know how frustrating the async programming model can be. Writing apps for lunatic can feel liberating. You get most of the performance, but don't need to worry about constantly keeping your future size in check and boxing them when needed, you can have recursive functions, and many other treats that make programming just a bit more enjoyable.