Very Future Thinking: Module Linking Relationship with Conditional Sections and Registry Enviornments
Closed this issue · 11 comments
Hello!
@lukewagner and I has a small chat about this where I expressed some concerns, but I thought maybe an issue would probably do it more justice than a long message haha!
So! One concern I have with the proposal (which may already be solved) is WebAssembly falling into the same trap of JavaScript concerning feature detection and syntax changes.
For example, in the JavaScript ecosystem there code is usually run on multiple platforms (Node and the Browser), and/or transpiled for different versions of those platforms (using something like Babel). This creates an interesting problem for library authors of public open source projects:
- Library authors who choose to support multiple platforms usually need to do some type of feature detection to figure out which platform their code is being run on, and create conditional statements for common tasks, or use an abstraction library to target multiple platforms (which leads to a lot of code bloat).
1a. To solve this, you can create multiple bundles to target each platform. However, this requires additional tooling as a part of your workflow. However, this still leads to some bloat as there usually is still some type of platform dependent checking either in the instantiation, or from using other dependencies target different platforms. The solution is not optimal, but it is acceptable and works! - As Javascipt (or any WebAPI) improves, not all browsers will implement or support the newest features. Thus this leads to code bloat either in feature detection, or using old APIs / syntax to support the most common use case (e.g SIMD would lead to smaller code size, but while SIMD is not supported, we'll have to ship larger code)
These two points (and others), lead to an interesting problem. Library authors become encouraged to ship bundles that match the least common denominator (in terms of feature support), as you don't know who/how your packages will be used downstream. Thus, this ends up leading to really bloated downstream libraries and applications that could be vastly improved by just removing unused code and more concise syntax of whatever version/platform the code is running on.
Your two options to solve this are:
- Hosting/offering a ton of different platform/version bundles. But this becomes difficult to maintain, and not choosing the right default becomes a hassle for users. And opting to choose the most performant/recent code usually leads to backlash from users who want your code to work everywhere on the web, not specific platforms/versions.
- Offer the source code, so that the code can be trans piled by the end library / application. But this means that the host bundler needs to support every feature that the library expects to be bundled away (e.g Dev vs. Prod code, TypeScript support, special assets imports, etc...). Which then becomes another big maintenance nightmare.
In the JS world, there was one proposal that a lot of people were really excited for: https://twitter.com/mathias/status/1105857829393653761 . The TL;DR that I understood of it was: "When a JS file / library is requested, you would know what "syntax year" of JavaScript was supported". Thus, library authors / developers could ship down the appropriate file that was optimized for that platform/version, without having to worry about things breaking. And all of the polyfills that were originally shipped down to detect and handle any platform/version differences are no longer needed because they know which features are supported before they start sending down bytes 😄
Going back to my concerns, essentially, it would let people building libraries know, what assumptions they can make before shipping down their code to users.
Thinking about how module linking relates, it seems like we can easily fall into the same problem of 1, since a lot of Wasm modules will probably want to support both WASI and browsers. But also, as wasm imporves and platforms aren't always the latest and greatest, conditional sections could lead to the same problem of 2, where we start shipping down a ton of unused bloated wasm code to polyfill new features as they get implemented.
With that being said, I was hoping we could look into how this problem can do this with module linking. And think about how this could be avoided as Wasm modules start to depend on one another, and be requested from platforms going forward? 😄 Perhaps it's a problem to be solved by tooling (such as Wasm Bundlers, or possible Wasm registries, etc...), or maybe something in wasm runtimes (such as requesting imported modules with headers stating what features this runtime supports/expects).
Thanks! Hope this isn't too long of a read, and my points may be easily debunked haha! 😄
cc @tlively since they champion conditional sections 😄
P.S One thing I was also curious about, in a general sense, how would things like tree shaking work? Would this be something to be solved by like a bundler? 😄
Thanks for the writeup, @torch2424!
One of the key ideas we have for conditional sections is that all feature detection in a wasm binary will be resolved at module compile time, rather than at run time. That means that it will be super clear from the structure of the binary what parts of the code go with what feature sets, which will make tree shaking super easy and will make it work the same way for all possible features. Instead of conservatively keeping code around when a bundler doesn't understand a feature, the bundler will be able to throw the code away, knowing that the module will still work correctly.
Hopefully the simplicity of that tree shaking will make it easy to implement in all bundlers, which will then allow library authors to include implementations for lots of different feature sets as conditional sections in their modules, giving their users maximum flexibility to target different feature sets in their downstream projects. I hope that tree shaking will make it easy for developers to create and host many specialized versions of their binaries to minimize downloaded code size as well. It would be cool if tools like Emscripten and wasm-bindgen could generate JS to do feature detection and automatically download the best version of the module for the end user's engine.
Another question you raised is how developers will know what feature sets to host binaries for. The caniuse-style table @RReverser added to https://webassembly.org/roadmap will hopefully make this easier for developers, but having a "syntax year" or some other way of naming a stable snapshot of WebAssembly features would certainly help as well. This is something we've talked about occasionally on the WebAssembly tools team at Google, but we haven't come up with any great ideas so far, so I'd be curious to hear what other people think. Part of the problem is that we only recently started stabilizing new WebAssembly features, so we don't have a lot of experience with the user story here yet.
As for how this all ties in to module linking, I think the best thing to do in the module linking proposal is to keep thinking about how conditional sections could be built on top of it as an extension and to leave room in the binary format for that extension. The discussion around that is mostly happening at WebAssembly/conditional-sections#22.
Thanks for the great question and summary of the issue! Agreed that conditional-sections#22 has the promising start of some potential solutions.
One idea I mentioned briefly at the end of my comments in conditional-sections#22 was allowing modules to be disabled based on the presence/absence of preceding modules -- that way you could avoid compiling the fallback code if you didn't need it. An example (riffing on my optional module examples in that comment) might look like:
(module
(optional module $featureA_test ...code that only validates if feature a is present...)
(optional module $hasA (condition (present $featureA_test)) ...code using featureA...)
(optional module $noA (condition (not (present $featureA_test))) ...code not using featureA...)
(module
... common code that links (somehow, TBD) to one of $hasA or $noA...
)
)Along the lines of what @tlively is saying, a tree-shaker could choose to target various profiles (for which it could anticipate whether $featureA_test would validate or not and then dead-code eliminate accordingly). But another approach that didn't rely on a build tool spitting out an a priori fixed set of variants (which I agree has downsides) would be to do it client-side, using the feature test to control which code was fetched.
For example, assuming that (condition ...) predicates could be applied to module imports as well, you could write:
(module
(optional module $featureA_test ...code that only validates if feature a is present...)
(import "./hasA.wasm" (condition (not (present $featureA_test))) (optional module $hasA))
(import "./noA.wasm" (condition (present $featureA_test)) (optional module $noA))
...
)If such a module was loaded with ESM-integration (or a polyfill thereof), then only the hasA.wasm or noA.wasm code would be fetched/compiled. An even smarter tool could choose whether to out-of-line (with module import) or inline (with nested module) based on size and balancing number-of-fetches with bytes-downloaded, all while only emitting a single version of the module.
@lukewagner does that (import "./hasA.wasm" ... come from the ESM-integration proposal? I haven't looked at that in a long time, but it would be super cool to be able to use it in conjunction with feature detection that way.
@tlively Yup! I sketch out how Module Linking + ESM-integration work in the FAQ. Then, if we added optional imports with conditions, I think the natural extension to ESM-integration is that the fetch is only performed if the condition is true.
(FWIW, you could also instantiate an (import "./hasA.wasm" ...) with the JS API; the key in the import object is './hasA.wasm. If the import's condition is false, then the JS API wouldn't lookup the key, but it's still up to the surrounding JS loader glue code to be smart about not doing unnecessary work.)
Yooo! Thank you so much for the thorough replies y'all! 😄 👍
One of the key ideas we have for conditional sections is that all feature detection in a wasm binary will be resolved at module compile time, rather than at run time. That means that it will be super clear from the structure of the binary what parts of the code go with what feature sets, which will make tree shaking super easy and will make it work the same way for all possible features. Instead of conservatively keeping code around when a bundler doesn't understand a feature, the bundler will be able to throw the code away, knowing that the module will still work correctly. Hopefully the simplicity of that tree shaking will make it easy to implement in all bundlers, which will then allow library authors to include implementations for lots of different feature sets as conditional sections in their modules, giving their users maximum flexibility to target different feature sets in their downstream projects. I hope that tree shaking will make it easy for developers to create and host many specialized versions of their binaries to minimize downloaded code size as well.
Ah that's great to hear! 😄 Glad to know things will be tree-shakable at build, as that I feel like is definitely going to be a key part in building the different highly-optimized modules users will want to build for each supported feature set.
It would be cool if tools like Emscripten and wasm-bindgen could generate JS to do feature detection and automatically download the best version of the module for the end user's engine.
Ah! So you're already touching on a point here, but I'll get into / propose later. I think the only person who knows how a library is going to be used, and in which environment is the end-application developer. I think it may be worth considering allowing the application developer tell libraries what they want when they request them, and not figure it out in the module itself.
Another question you raised is how developers will know what feature sets to host binaries for. The caniuse-style table @RReverser added to https://webassembly.org/roadmap will hopefully make this easier for developers, but having a "syntax year" or some other way of naming a stable snapshot of WebAssembly features would certainly help as well. This is something we've talked about occasionally on the WebAssembly tools team at Google, but we haven't come up with any great ideas so far, so I'd be curious to hear what other people think. Part of the problem is that we only recently started stabilizing new WebAssembly features, so we don't have a lot of experience with the user story here yet.
Oh yeah! @RReverser did an awesome job on that table 😄 And definitely these types of tables will be important for maintaining part of the flexibility of the web (similar to JS).
But yes! You see my point, if you have "snap shots", instead of including a ton of code switching over 10 different features, you could have at most one and then assume the other 10.
And yes, I definitely agree WebAssembly is just baaaarreeelllyyy starting to rub up on this issue. But I feel like we should definitely think about it soon-ish before it's set into the ecosystem 🤔 I also think we are already in really good position to start figuring this out if it's a good idea, as the communitty is still small-ish, and well-organized I think 😄
As for how this all ties in to module linking, I think the best thing to do in the module linking proposal is to keep thinking about how conditional sections could be built on top of it as an extension and to leave room in the binary format for that extension. The discussion around that is mostly happening at WebAssembly/conditional-sections#22.
Awesome! I'll take a look there and get some more context 😄
One idea I mentioned briefly at the end of my comments in conditional-sections#22 was allowing modules to be disabled based on the presence/absence of preceding modules -- that way you could avoid compiling the fallback code if you didn't need it. An example (riffing on my optional module examples in that comment) might look like:
Ah! This is awesome to see! Glad that we have the idea of optional modules, that aren't requested unless they are truly needed 😄
Along the lines of what @tlively is saying, a tree-shaker could choose to target various profiles (for which it could anticipate whether $featureA_test would validate or not and then dead-code eliminate accordingly). But another approach that didn't rely on a build tool spitting out an a priori fixed set of variants (which I agree has downsides) would be to do it client-side, using the feature test to control which code was fetched.
Ah yes, so I think this also touches on the point which I'll just go ahead and get into:
Awesome! So thanks again for taking the time to read and reply to my essay haha! 😄
So I definitely really happy to see that at build time we do have the ability to produce modules that for the most part, will be able to tree shake away code that will not be used for whichever features / feature set we compile it for. And for whatever we don't know at build time, we can use runtime feature detection to grab the best module for our use case.
However, I still think does not solve the problem if I am not mistaken. As this this how things essentially work in JS today. I guess I can cut to the chase and kinda expand on the solution I was thinking about:
Just as some context, in general I kind of see runtime feature detection as like something that we have to do, but it's not necessarily a good thing, as the more features you need to detect, the more bloated your code will become (generally). And another thing, I don't think developers are going to always follow the best case scenerio is going to be the most common unfortunately. But I may be wrong 😄
So as a (good) library author, you can totally at build time, try and generate as many modules for as many different feature sets you want to support. However, the only person that truly knows what platform they are going to be running on is the application developer. Which I think encourages runtime feature detection somewhere along the chain.
So for example, let's say I want to build a wasm module that runs on both the browser and WASI (X), and it imports anther wasm module (Y). Thankfully, (Y) offers both a WASI and browser version, but (X) forgets to pass a flag or something like that to their compiler, well then (Y) has to do some runtime feature detection in like some "default index module" to then grab the appropriate module. And depending on the size of the module, it may be better to just ship everything down rather than make two round trips.
What I'm really trying to get to is (sorry for all the context): Why don't we allow the end user to pass along context when it imports modules?
Like what if we did something like:
(module
(import "./hasA.wasm" (context "WASI" "SIMD" "THREADS"))
...
)
And then this can be sent along as headers (or through some global variable in a registry / bundler) or something to whatever is fufill the dependency. And now they can make the decision of which module to send (Kind of like src-set / syntax year proposal). Rather than the library having the figure this out.
And let's imagine a scenario of a library with like 10 layers deep of dependent libraries. Not all of them will be properly updated as new features come out, but maybe the deepest layers will be maintained. It would be cool if this context could be shared with every layer, so every module could be resolved to it's most highly optimized feature set along the chain. So when layer 3 doesn't update layer 5 because it stopped being maintained, layer 5 can still ship down it's most optimized code. I also think this general idea will be super helpful to like CDNs and things, if instead of having to runtime feature detect which module to grab, we could just tell the CDN "this is my assumed environment give me the best one please" 😂
Am I making sense? The TL;DR Being: I think the problem of figuring out feature sets for an application should be on the end-user building the application and whatever is resolving dependencies, not the library authors on top of other library authors. In the sense that we want to give library authors the tools to highly optimize their modules for specific use cases, but not incentives bloated code at runtime for portability
Let me know what you think! 😄 👍
@torch2424 Thanks for explaining and I appreciate you focusing on both the concerns of library devs and of app devs and trying to empower both to do the right thing easily.
So I think I see two situations in your comment that I'll try to consider in order since I think (2) could build on (1):
- at application build time, I want to specialize my tree of dependencies without relying on the individual dependencies to have already specialized themselves
- at client request time, I want my client's specific feature set to specialize what wasm code is sent down to the client
For (1), I may be missing some constraints or requirements, but I think the optional module scheme mentioned above should achieve what you want: if we assume each module in the dependency tree ultimately makes its own module-local decisions based only on (a) validating nested optional tester modules, (b) optional imports supplied by the client, then nothing is committed to when publishing a library. Rather, at application build time when I know all the modules and how they are linked, I can tree-shake my entire dependency tree by telling the tree shaker which features should validate (e.g., in terms of feature flags to wabt that are passed when validating all the optional nested modules) which would then allow the tree-shaker to eliminate dead code.
For (2), that sounds a feature of the CDN wherein the client does some feature tests (maybe with an inline <script> using WebAssembly.validate(Uint8Array.of(...inline bytecode...)) so it can happen Really Early(tm) in pageload) and sends these feature-test results as CDN-specific HTTP request parameters. The CDN could then use these request parameters to send down a specialized (tree-shaken) wasm module (either from a statically pre-computed set, or even doing it dynamically with caching).
How does that sound? Happy to hear any requirements I'm missing or failing to address.
So I think I see two situations in your comment that I'll try to consider order since I think (2) could build on (1):
Yep! These are it exactly! 😄
For (1), I may be missing some constraints or requirements, but I think the optional module scheme mentioned above should achieve what you want:
Oh yeah definitely! So I think/know this would 100% solve the problem of "if we can" do it. It's more than, I'm thinking of the perspective of how can we do it while shipping down the least amount of bytes possible to the client in an environment where this is not being done in the "best case scenario" way 😄
if we assume each module in the dependency tree ultimately makes its own module-local decisions based only on (a) validating nested optional tester modules, (b) optional imports supplied by the client, then nothing is committed to when publishing a library.
So this may actually be something I don't understand about JS Transpiliation vs. general compilation. So please excuse me if I'm not making sense here 😂 But I think really, my whole concern here just really revolves around (a). I don't want to ship down the bytes for (a) if I don't have too. Especially if we can imagine having nested modules that are all all checking for the same (a) these could add up really fast.
So in JS, (a) can't really be compiled away. As there's not really an easy way to detect when/where/why they are testing for features, so if the feature is supported, only the "top-level app code" can transpile it away, but you can't really for the dependencies (like 90% of the time). And what makes this kind of worse, a lot of times to save bytes, applications will just inline the "older solution" rather than both when they offer their module (but it sounds like, wasm probably wont have this problem at least if we can encourage not doing this to compilers because conditional sections will be tree shaken away?)
So in Wasm, is (a) going to be able to be compiled away? I think it would be hard for a compiler linking Wasm modules to know what those conditions are without actually running it in the environment, right? Especially if you think about deep levels of nested old with modern Wasm code that will eventually have to work together. Which is why I'm kind of suggesting, could we tell our wasm compiler/linker what those conditions would evaluate to for our expected use case? Then it could tree shake everything away 😄 And that's if we can assume every (popular) compiler compiling Wasm will correctly use (a) with conditional sections. 🤔
Rather, at application build time when I know all the modules and how they are linked, I can tree-shake my entire dependency tree by telling the tree shaker which features should validate (e.g., in terms of feature flags to wabt that are passed when validating all the optional nested modules) which would then allow the tree-shaker to eliminate dead code.
Ah, so after reading this again, I guess that solves my Wasm with (a) correct? 🤔 😄
For (2), that sounds a feature of the CDN wherein the client does some feature tests (maybe with an inline <script> using WebAssembly.validate(Uint8Array.of(...inline bytecode...)) so it can happen Really Early(tm) in pageload) and sends these feature-test results as CDN-specific HTTP request parameters. The CDN could then use these request parameters to send down a specialized (tree-shaken) wasm module (either from a statically pre-computed set, or even doing it dynamically with caching).
Oh yeah! So that would totally work, and work well. But I'd want to avoid that second round trip, and not have to ship down testing code.
One thing I may also be missing here, do we expect wasm module imports to make requests when running in a browser? Or is (import "./my-module.wasm") a bundler only feature? But then again, looking back at the "Wasm with (a)" if we can send our compiler/bundler the hints, it could remove all the testing code in (a) and just give the final URL we wanted? 😄
How does that sound? Happy to hear any requirements I'm missing or failing to address.
It all sounds great! I think we may be on the same page here, especially after hearing your next response 😄 And I think we got the "save as many bytes as possible while bundling apps" requirements down now haha! 😂 😄
Ah, so after reading this again, I guess that solves my Wasm with (a) correct? 🤔 😄
Yes, I think so, at least for core wasm feature testing. The other side of the coin is optional imports of host (Web) APIs. For these, the tree-shaker needs some extra information to be able to statically specialize on whether the optional import was present or not. Now, my original hope was that, with the revised get-originals proposal, Web APIs could be imported declaratively-enough such that all you had to know about was the import-map (which can polyfill/censor/virtualize originals). Even without that standardized proposal, there could still be a toolchain convention of naming wasm optional imports with the platform APIs they are importing such that the a tree-shaker could be told statically what it could assume was present.
Oh yeah! So that would totally work, and work well. But I'd want to avoid that second round trip, and not have to ship down testing code.
My (possibly wrong) assumption here is that the bundler could emit a tester script which could be just a handful of bytes (such that download size impact is negligible) that gets placed early in the HTML document (so there are no extra roundtrips or latency when fetching the .wasms).
One thing I may also be missing here, do we expect wasm module imports to make requests when running in a browser? Or is (import "./my-module.wasm") a bundler only feature?
I think ultimately this is up to the bundler, so yeah, a bundler would be able to arbitrarily replace all import "./my-module.wasm"'s with (module ...contents of my-module.wasm...)'s, if it wants to.
But then again, looking back at the "Wasm with (a)" if we can send our compiler/bundler the hints, it could remove all the testing code in (a) and just give the final URL we wanted? 😄
Yep, if you turn up the specialization to the max, then all you would need is that single early-inline-feature-test script I mentioned above; everything else could be specialized based on that, I think.
@lukewagner Thanks for the reply! I tried to keep this one shorter. And it really is, but I make one huge quote seciton which makes it look big haha! 😂
Yes, I think so, at least for core wasm feature testing. The other side of the coin is optional imports of host (Web) APIs. For these, the tree-shaker needs some extra information to be able to statically specialize on whether the optional import was present or not. Now, my original hope was that, with the revised get-originals proposal, Web APIs could be imported declaratively-enough such that all you had to know about was the import-map (which can polyfill/censor/virtualize originals). Even without that standardized proposal, there could still be a toolchain convention of naming wasm optional imports with the platform APIs they are importing such that the a tree-shaker could be told statically what it could assume was present.
Awesome this part all sounds great, glad we got two plans going forward here 😄 Thank you!
Next, I'm going to reply to this whole section of question/replies:
Oh yeah! So that would totally work, and work well. But I'd want to avoid that second round trip, and not have to ship down testing code. (Reply) My (possibly wrong) assumption here is that the bundler could emit a tester script which could be just a handful of bytes (such that download size impact is negligible) that gets placed early in the HTML document (so there are no extra roundtrips or latency when fetching the .wasms).
One thing I may also be missing here, do we expect wasm module imports to make requests when running in a browser? Or is (import "./my-module.wasm") a bundler only feature? (Reply) I think ultimately this is up to the bundler, so yeah, a bundler would be able to arbitrarily replace all import "./my-module.wasm"'s with (module ...contents of my-module.wasm...)'s, if it wants to.
But then again, looking back at the "Wasm with (a)" if we can send our compiler/bundler the hints, it could remove all the testing code in (a) and just give the final URL we wanted? smile (Reply) Yep, if you turn up the specialization to the max, then all you would need is that single early-inline-feature-test script I mentioned above; everything else could be specialized based on that, I think.
So I asked these two questions because they I think they paint a picture together 😄 I'm imagining this scenario:
We get a module in our browser that has optional imports, and includes a tester script for threads support. So the module runs, and we find out we don't have threads support, so we now need to make a second round trip to the server to go and grab import that doesn't support threads.
In general multiple round trips are really hurtful for load times. To avoid this second round trip we could just inline both versions of our optional module, but this depends if the module little enough to be less expensive than a second round trip. Or we could HTTP Push down both versions (which avoid the second round trip, but sends unneeded bytes). So in both cases, where the bundler inlines the module, or in the case where in the browser we are requesting dependencies on the fly, having a tester script is going to be hurtful since we can't ship down only the bytes that our specific platform, in this one instance of the many, needs.
Which kind of gets back to the core of what I'm hoping we can try to do (doesn't have to be this exactly, but the same idea):
On request time, our browser/runtime knows what it supports. It would be awesome then when request that original module, before any bytes at all are shipped down, we could tell them: "Hello server! I want this module. Just a heads up, I support threads, but I don't support SIMD. Please consider that information before you send me back the module I want."
That way, the end-user application / library developers can make multiple builds for every feature configuration they want to support. And whatever is responding to that request at runtime for the module or the dependency or whatever it may be, can use that information to serve back down the most optimal module using the information the requester provided.
We have agreed that the current state of wasm / plans for wasm will allow a compiler will make sure the bundler will use the most optimal dependencies all along the tree (which is great, solves probably the biggest worry I had, and makes the non-browser use case entirely awesome). Therefore we will be able to build these highly-optimized nearly bloat-free dependency tree modules. But I'm still a little concerned we won't know when each should be served at request time.
Lastly, on top of the tester scripts forcing requests to be made in second round trips, so on and so forth. When you say the tester scripts are a handful of bytes, do you really mean ~5 bytes? I'd image they'd be bigger, but in the grand scheme of things, yes they would be really small. But I do think they add up. Just to give some perspective from me: Each kilobyte shipped down is about 33 milliseconds added to a 2G load time, doing the math from here: https://bundlephobia.com/result?p=responsive-gamepad@1.2.1 . 140ms / 4.2Kb gives us 33 milliseconds. I'm sure we can both imagine a world where a total application may check for 10 - 20 wasm features in the next 3 years (just an estimate). Even if each testing script is only 100 bytes, that's a bit of time added to each request on 2G networks which is common in emerging markets (and in my opinion, the markets that probably need wasm the most 😂 ). (On a second read, let me know if you are implying a single tester script. if it was a single script, this point still stands unless it's possible for a single script that matches over all of these features to only be a handful of bytes)
But yeah, let me know what you think! Also, sorry if I am talking in circles. I'm on vacation this week, so my brain is still on vacation mode, while I try to piece the puzzle together if my worries are already solved or not 😂
@torch2424 Cool, that makes sense. My last answer about the "tester script" was a bit hand-wavy and incomplete, so let me expand on it a bit more because I still have some hope that it addresses the concern you're describing here without any native browser support.
So let's say that my bundler ultimately wants to specialize my application pageload time based on some space of options S. S could simply be a Cartesian product of every independent wasm feature, or it could be something coarser granularity to capture just the N different configurations that are meaningfully-different in the wild. Whatever the case, the bundler should be able to spit out a tester script that contains 1 or more WebAssembly.validate(Uint8Array.of(...))s (and ad hoc JS feature tests) that cover S. The number of bytes to test a single independent wasm feature should be, in most cases <100 bytes because, when the only thing you care about is "Does It Validate?", wasm allows some fairly small binaries. E.g., wabt encodes (module (memory 1 1 shared)), which you could use to test for threads, in 14 bytes, which will expand somewhat when inlined into the Uint8Array.of(...), but I guess you could optimize that with a Uint32Array.of(...) ;)
If this tester script is included in the <head> of the initial .html that the browser fetches (before it can even possibly fetch the first .wasm) then the results of the tester script can be used to craft the URL of the initial .wasm fetch, so I think the net result should be no extra round trip with no extra bytes sent down.
Another advantages of this approach would be you could build and deploy it today. Also, it would allow the toolchain, not the browser to decide how to slice and dice the option space S. Otherwise, the browser is going to have to make the uncomfortable tradeoff of a fine degree of feature granularity (increasing the number of often-ignored bytes included in HTTP requests) and a coarse degree of feature granularity (which minimizes request overhead but may not be precise enough for the bundler's needs). It also lets the bundler do totally ad hoc tests like "does the browser have bug X?" which would otherwise require the browser to start advertising "no, I don't have bug X".
@lukewagner Ahhhh! Thank you for the thorough explanation! 😄 I can definitely see how this will be a bit different than the current state of JS, and I think this is a good solution, especially because the point you mentioned about browser bugs.
And yep, including the tester script in the head of the HTML sounds like a great plan to encourage and things.
All in all, thank you very much for explaning this to me, and I am super glad to hear we have a solid plan on how we should encourage toolchains to handle this issue.
Closing the issue, thank you! 😄 🎉