Investigate ZombieNet Integration Requirements
Closed this issue ยท 5 comments
The outcome of this Issue should be a well defined list of the APIs and integrations that Gossamer must implement in order to allow Gossamer nodes to be included in ZombieNet networks and testing pipelines.
Notes:
- ZombieNet has two primary functions: spawn and test.
- Spawn starts ephemeral networks based on configurations defined by the network definition file (toml or json).
- Test provides a way to conduct test/assertions on the nodes in the networks. Tests are constructed using the ZombieNet DSL which handles interacting with the using polkadot.js api, metrics and logs.
I've started investigating requirements necessary to spawn gossamer nodes. The network definition spec defines attributes for defining a nodes configuration. A network is defines a provider that will host the network: kubernetes, Podman or native. I've started testing using native provider, which is starting the network on local hardware.
When executing the spawn
command ZombieNet, the default behavior is to call build-spec
to build a raw (machine readable) chain spec (based on chain
attribute in network definition) to use when starting the nodes. The attributes sent to the build spec command for polkadot nodes are different than those used with gossamer's build-spec command, so using the default behavior with gossamer nodes does not work. Using the chain_spec_path
attribute overrides this behavior by using the chain spec defined by the file set in chain_spec_path
and does not call the build spec command. Using this does work with gossamer nodes.
The default_command
attribute defines the commands that will be executed to start nodes, the default_command also starts the node with a set of pre-defined default arguments that are passed into the default command. Some of these defaults arguments are arguments that the gossamer node does not recognize, so the gossamer node does not start when using the 'default_command' or 'command' attributes due to errors of unrecognized CLI arguments.
Using the attribute command_with_args
solves this issue since using this command there are no default arguments that are included. Only the args that are defined with the attribute are passed to the command when starting.
Using chain_spec_path
and command_with_args
will start gossamer nodes, however ZombieNet does not realize the nodes have started, and therefore stops the nodes after a one minute timeout period. Starting ZombieNet with debugging logging set (using DEBUG=zombie* ./zombienet-linux-x64 spawn --provider native examples/0001-small-network-gossamer.toml
) we can see that ZombieNet uses bash grep to inspect the node's logs searching for regular expression 'Listening for new connections|Running JSON-RPC'
to determine if a nodes has started. Since gossamer nodes do not report that when it's started using either of these terms, ZombieNet assumes the nodes has not started. I've created issue #2998 suggesting we update gossamer logs at startup to log one of these terms to the log file.
Notes on modifications made to zombienet cmdGenerator to start gossamer nodes:
- defalut flag for polkadot
--no-mdns
for gossamer its:--nomdns
- default flag for polkadot
--node-key
for gossamer it's--key
- the key value passed into this is generated by zombienet: TODO: learn how this works, test
- currently testing with test values, alice, bob,...
- default flag
--prometheus-port
doesn't work with gossamer, TODO: determine flags for prometheus - default flag
--rpc-port
for gosamer it's--rpcport
- default flag
--ws-port
for gossamer it's--wsport
- default flag
--listen-addr
doesn't work with gossamer - default flag
--base-path
for gossamer it's--basepath
- default flag
--rpc-cors' value
all` doesn't work with gossamer - default flag
--unsafe-rpc-external
doesn't work with gossamer - default flag
--rpc-methods
valueunsafe
doesn't work with gossamer - default flag
--unsafe-ws-external
doesn't work with gossamer - default flag
--validator
for gossamer it's--roles
value4
Testing DSL with javascript, example script is failing with error -32000: rpc: can't find method "state.QueryStorageAt": null
Cross client testing, testing to run gossamer node with polkadot node, current configuration works to start different node, but the nodes do not communicate with each other. Zombienet uses --node-key
flag to define lib p2p network addresses for the nodes, then uses that address to build parameters for --bootnodes
and --listen-addr
(polkadot cli argument to set listen address). Zombienet builds the addresses in the form
/ip4/127.0.0.1/tcp/45475/ws/p2p/12D3KooWQCkBm1BYtkHpocxCwMgR8yjitEeHGx8spzcDLGt2gkBm
however default local listen addresses are in the form:
/ip4/127.0.0.1/tcp/30333/p2p/12D3KooWRkZhiRhsqmrQ28rt73K7V3aCBpqKrLGSXmZ99PTcTZby
Note that the zombienet created address includes /ws/
after the port number, while the default addresses do not, this addition is handled by the --listen-addr
flag since zombienet uses the address with the /ws/
when defining the new listen address. Gossamer nodes do not have the --listen-addr
flag, so when the --bootnodes
flag with the address that contains /ws/
is used the gossamer nodes are not aware of them.
Implementing the --listen-addr
flag to gossamer would solve this issue, or zombienet could be modified to build different boot node addresses for different nodes.
๐ This issue has been resolved in version 0.8.0 ๐
The release is available on GitHub release
Your semantic-release bot ๐ฆ๐