Re-think Test Fixture Gen / Approach
harrysolovay opened this issue · 3 comments
Currently, test fixtures are being generated as WASM + bindings. This allows us to directly utilize the SCALE Rust crate, which is the best source of truth, behaviorally. However, this introduces some unfortunate complexity to the codebase.
I know Rust is cool and all, but I really don't think it's a good idea to use Rust.
The SCALE format is stable. It's never going to change. Hardcoding some encoded values instead of building them every time the tests are run is completely safe.
Using Rust brings a lot of complexity to the repo, for really no gain whatsoever over hardcoded values.
Originally posted by @tomaka in #1 (comment)
^ how do we want to tackle this?
I've been thinking about this a bit.
On the one hand, using Rust adds a bit of complexity to the repo, as pointed out earlier. The CI, for example, needs to pull in all of rust and wasm-bindgen, which, when uncached, makes the total build time around 5 minutes.
On the other hand, the encoded values have to come from somewhere, at least initially.
I propose a middleground: In the TypeScript tests, use snapshots for testing the encoded values, with an adjacent file with a list of equivalent Rust values for each of the snapshots tested in the TypeScript file. Then, have two separate test scripts: one which runs the TypeScript tests (and completely ignores the Rust values), and one which checks the Rust values against the snapshot files.
This way, one can do a partial check without having Rust installed, and without generating any fixtures (as the snapshots of encoded values would already be committed), but the snapshots could still be double-checked against the Rust implementation (in, say, CI). The Rust testing could be done natively (i.e. not through WASM), which would simplify running it.
Thoughts?
Yes! If I'm not mistaken, we can address this by simply un-gitignoring and then committing the generated WASM and bindings?
Hmm... I suppose so, though I had a slightly different approach in mind, that I think might make the setup a little simpler and cleaner. Mind if I experiment with an alternate approach?