libp2p/test-plans

[tracking issue] Interop test-plans for all existing/developing libp2p transports

p-shahi opened this issue · 13 comments

p-shahi commented

Part of the libp2p/test-plans roadmap: https://github.com/libp2p/test-plans/blob/master/ROADMAP.md#2-interop-test-plans-for-all-existingdeveloping-libp2p-transports

Done Criteria

Using tooling from #53

  • All features of go-libp2p, rust-libp2p, and js-libp2p that should be interoperable have test-plans (i.e. transports (TCP, QUIC, WebRTC, WebTransport), multiplexers (mplex, yamux), secure channels (TLS, Noise), etc.) across versions.
  • Features currently in development across implementations (like WebRTC in go-libp2p and rust-libp2p, or QUIC & TLS in rust-libp2p) are not merged without at least manually running these test suites.
  • Test suites are run in libp2p/test-plans CI and before releasing a version of go-libp2p, rust-libp2p, and js-libp2p (GitHub workflow added so that these suites run against the master branch on a nightly basis (updating the status check.))

Tasks

John-LittleBearLabs commented

I don't have permission to edit this issue, but I believe it was suggested that we add #82 as a task here.

  • 👍1
GlenDC commented

In regards to js-libp2p (node/browser) #74, as in accordance with @laurentsenta we agreed on the following approach moving forward:

  1. Implement the libp2p ping test using node.js, with just two nodes that can communicate with each other.
  2. Add the test to the composer, using @Tanguy (nim-libp2p)'s recent merge as a guide. This will still be a single node.js version added to the test (https://github.com/libp2p/test-plans/pull/70/files)
  3. Expand the configuration to include browser testing.
  4. Tackle the challenge of dealing with many versions.

It might make sense to first aim for (1-3) using the latest release of libp2p-js, and do (4) in a secondary PR.

BigLep commented

Thanks for the update @GlenDC . I like the plan to do this incrementally. Can 1&2 be the first PR (just using node), then do 3, then do 4?

GlenDC commented

Hi @BigLep. The goal is definitely to do it incrementally, however given we already the browser+node support in testground thanks to the fact that we start from https://github.com/testground/testground/tree/master/plans/example-browser-node, the plan that we created as an example of how to test plans without modification in both a node and browser environment. For now we have to copy from the plan, but thanks to real-world use cases such as this libp2p-js one, we gather insights on all the requirements and catcha's. Based on that knowledge it will be turned into something thats built-in support.

For the time being we will however need to copy the example as a base, which is what I did already in my draft PR (unfinished for now) and which is why the diff is so big. But in reality the things that are unique to this PR are only the testplan found in the /src directory and the added dependencies to the package.json.

As such my proposed steps are:

  1. get the libp2p ping test working between two nodejs instances using a static configuration (so not too fancy in how we configure the libp2p node, but just do it using hardcoded noise encryption and websockets transport).
  2. with this in place we can also add a test between nodejs and the browser runtimes (chromium, firefox and webkit)
  • given libp2p is supposed to already be browser-ready this should just work out of the box without any additional effort beyond adding the testplan toml
  1. with all this working we should be able to add the 4 runtimes for libp2p-js (nodejs, chromium, firefox and webkit) to the same composition files as the Nim PR did recently.

Step (1), (2) and (3) are planned by me to be all done in the first PR, which I aim to have ready next week for review. Given that all dev effort goes into (1), it should be ok. To keep things simple this will all be done using the same libp2p-node config (construction) and it will be all done using he latest npm version of libp2p-js.

In a second PR (once the currently discussed PR is approved and merged) I then plan to work on step (4):

  1. Tackle the challenge of dealing with many versions.

In that same (second) PR or a different PR we can also look into allowing tests to be run using different configurations. E.g. also support WebRTC / TCP transports, etc... But not sure what the requirements there are. That would essentially be a step (5).

p-shahi commented

@GlenDC @MarcoPolo:
Glen, do you plan to pick up the WebRTC tests and add them to docker compose as well? I've closed #67 and #84

GlenDC commented

Glen, do you plan to pick up the WebRTC tests and add them to docker compose as well

@p-shahi I could and will do with pleasure. Here are my proposed next steps:

  • this week: my first step (now that we have JS node test working for ping: https://github.com/libp2p/test-plans/tree/c35de22/multidim-interop/js/v0.41) will be to add decent dual-runtime support (JS+browser) so we can reuse that code for all versions and js tests (not just v0.41 ping), you can expect PR later this week;
  • this week or next week: add WebRTC support to ping test (I suppose you mean that?)
  • if you mean to port the WebRTC tests that Little Bear Labs developed in a separate repo then I can do that as well (this week or next week, but that might take a bit longer depending how well that porting goes, especially as the JS ones would be te first to add it here?)
p-shahi commented

Thanks @GlenDC your plan sgtm.

add WebRTC support to ping test (I suppose you mean that?)

Yep

if you mean to port the WebRTC tests that Little Bear Labs developed in a separate repo then I can do that as well (this week or next week, but that might take a bit longer depending how well that porting goes, especially as the JS ones would be te first to add it here?)

Great idea here. We definitely want those in test-plans.

jxs commented

Hi @p-shahi I can continue the work on #84 and add WebRTC support to the Rust version, (sorry wrongly assumed this was already done on #97)

  • ❤️1
p-shahi commented

That sounds good @jxs thank you!

p-shahi commented

If this sounds good to Marco,
@jxs will you be able to take up the Rust complement of libp2p/go-libp2p#1991
@GlenDC for JS?

GlenDC commented

With some guidance @p-shahi , sure. To be done after my current PR in progress, or after everything else we already discussed?

jxs commented

yeah no worries @p-shahi I will look into it 👍

MarcoPolo commented

Sounds good. If folks need any help with getting the GitHub actions integration working lmk. I’ve already spent a fair bit of time to get this working.

  • 👍1