Creating a executer agnostic crate via Nuclei?

[d4h0]

Hi,

have you seen Nuclei from the Bastion project?

Wouldn't it be possible with Nuclei to create an executer agnostic version of this crate? Or at least, wouldn't it be possible for users of this crate to create a crate that doesn't depend on a specific executer?

Nuclei supports (via Agnostik): Bastion, Async-Std, Tokio, and Smol – so basically all major executors, it seems.

I still try to wrap my head around all this, but wouldn't it be possible at the moment to create a TCP socket to the WebSocket server using Nuclei, and then pass that socket/stream to client_async?

Are there any disadvantages to this approach?

I think it would make it much easier to create executer agnostic crates.

At the moment, it seems, I have two options with async-tungstenite:

A) I let the user create the WebSocket / require a function as an argument that creates the WebSocket (and use Agnostik or async_executors as an abstraction over the executors).

B) use feature flags to implement executer specific code

With Nuclei, it seems, it would be possible to just request a supported executer and use a WebSocket that uses a Nuclei TCP Stream.

What do you think?

[sdroege]

Or at least, wouldn't it be possible for users of this crate to create a crate that doesn't depend on a specific executer?

Currently async-tungstenite does not depend on any specific executor, but provides optional integration into async-std, tokio and gio.

I saw Nuclei but didn't look closely yet. I think it would be possible to add Nuclei support to async-tungstenite similar to the existing async-std/tokio/gio integration. If you want to give it a try I'd be happy to review a PR for that. It would basically work how you describe above: creating a TCP connection somehow and then passing that to client_async. That's more or less all the existing executor/runtime integrations are doing too.

[d4h0]

Currently async-tungstenite does not depend on any specific executor, but provides optional integration into async-std, tokio and gio.

Yes, but to use async-tungstenite it's still required to create some executer specific code, right?

If you want to give it a try I'd be happy to review a PR for that.

Okay, I will most likely give this approach a try, and if I don't run into any problems I'll send a PR.

[sdroege]

Yes, but to use async-tungstenite it's still required to create some executer specific code, right?

Yes, but that could be any :)

Okay, I will most likely give this approach a try, and if I don't run into any problems I'll send a PR.

If you run into problems, feel to ask here

[PhotonQuantum]

@d4h0 Do you have any progress on implementing the nuclei adapter?

I tried to implement it, but can only make it work on iouring. Both epoll and kqueue fail at handshake stage, due to the socket being stuck when reading. I suspect there's something wrong with nuclei.

[d4h0]

@PhotonQuantum: Oops, I completely forgot about this issue... 😆

As far as I remember, there were several reasons why I gave up on this:

1. It was pretty easy to implement (5 lines of code), so there wasn't much value in providing this in async-tungstenite, I thought (especially, when considering the next point)

2. It is not really an ideal solution (as I've realized after creating this issue). Unfortunately, I don't remember the details anymore. I think the problem was, that the end-result was that there could be several IO backends/runtimes, or something like that. Less bad than several executors, but still not ideal. Basically, from a performance point-of-view it wasn't perfect (but today I think, in most situation it would be more than good enough. But maybe not ideal in a library crate...).

It seems, I've used nuclei with the bastion and epoll features. If you are testing with another runtime, maybe give bastion a try (to test if the runtime might be the problem).

Besides that, I basically just had the following:

    nuclei::spawn_blocking(|| nuclei::drive(future::pending::<()>()));

    match nuclei::block_on(run()) {
        Ok(_) => info!("Successfully exited"),
        Err(e) => error!("Failed with: {:#?}", e)
    };

Personally, I gave up on executor agnostic code, for now. Async Rust is already often pretty painful on its own, so I don't want to make it worse by trying to write code that runs on every runtime. At some point we will get the required traits and types to build an "executor agnostic async ecosystem". Until then, I'll just focus on tokio.

@sdroege: I'll close this issue because of what I wrote above. Feel free to re-open, if you want to add nuclei support to async-tungstenite.

[sdroege]

I'll close this issue because of what I wrote above. Feel free to re-open, if you want to add nuclei support to async-tungstenite.

If it's actually useful for someone and they create a PR then I'd be happy to merge that