Representing module linking in terms of ESM integration

[littledan]

This proposal is based on the idea that a module may be run multiple times, with the same import being filled in different ways. The use cases for this goal seem convincing to me. I'm trying to think of how it could be represented in terms of Wasm/ESM integration, where modules are only instantiated one time within each Realm.

In ES module environments, the module specifier is used with a particular interpretation for where to get the module from the outside world (rather than as a parameter): On the web, module specifiers are treated as URLs, and in Node.js, as paths--neither of these gives a clear way to treat them as arbitrary parameters which can be filled in different ways on different instantiations.

A couple ways that we could bridge this gap that come to mind to me:

• If a module takes another module in a potentially varying way (as noted in a new custom section for ESM integration?), then in ESM, it is exposed as a function which expects a module namespace object as a parameter, and returns another module namespace object (of exports).
• Somehow, an API based on Realms could be created, to allow some modules to be filled in in a certain way, and then other modules could be instantiated in that Realm.

In either case, it's not really clear to me how to "wire up" complex module graphs as in the various case studies described in this repository. It sounds like it would take some significant application-specific loader code.

Do you have any thoughts on how to connect these proposals?

[lukewagner]

I don't know if you saw it, but I tried to give an answer to this question in the FAQ. In the context of that FAQ answer, do you have any further questions?

[littledan]

Ah, I'm not sure how I missed that section!

A big piece that I was missing above, that is explained clearly in that FAQ, is how this proposal makes it clear, at the import site, based on the type, whether the intention is to import the module or instance, so modules can be treated differently. That's a key addition to expressiveness that we don't have in JS (but maybe could with evaluator attributes). I like this aspect of the design.

Is the idea here that inner nested module declarations, as well as imports of modules (rather than instances), would only be able to instantiate themselves with the private mappings created in the module linking proposal?

Would it be limiting to treat modules as all-or-nothing, about whether they are resolving with respect to the external module map, or the private mapping for this particular set of modules? I'd imagine that, sometimes, you want to pass around just the WASI capabilities, but let modules import other, external things.

[lukewagner]

Is the idea here that inner nested module declarations, as well as imports of modules (rather than instances), would only be able to instantiate themselves with the private mappings created in the module linking proposal?

Yup!

Would it be limiting to treat modules as all-or-nothing, about whether they are resolving with respect to the external module map, or the private mapping for this particular set of modules? I'd imagine that, sometimes, you want to pass around just the WASI capabilities, but let modules import other, external things.

You're right, that does seem useful. Could evaluator attributes + dynamic import() allow this? It seems like there would be two modes: (1) supplying a set of imports that are the only imports available to the module (like WA.instantiate(); good for sandboxing use cases), (2) supplying a set of imports that are consulted first, before the external module map.

[littledan]

Would it be limiting to treat modules as all-or-nothing, about whether they are resolving with respect to the external module map, or the private mapping for this particular set of modules? I'd imagine that, sometimes, you want to pass around just the WASI capabilities, but let modules import other, external things.

You're right, that does seem useful. Could evaluator attributes + dynamic import() allow this? It seems like there would be two modes: (1) supplying a set of imports that are the only imports available to the module (like WA.instantiate(); good for sandboxing use cases), (2) supplying a set of imports that are consulted first, before the external module map.

I'm getting a bit confused; let's talk about the Wasm + HTML case before we bring JS into it. I'm imagining that you can use Wasm/ESM integration without (much?) JS by including Wasm directly in a script tag.

In this case, you may have a module graph of Wasm modules, where some are included inline, and others are intended to be interpreted as separate, external HTTP requests. Should these be allowed together, or should you always have to choose between 100% inline modules or 100% external modules? (I don't see how this would be covered by 1 or 2.)

(I don't really see how sandboxing would be accessible ESM integration at all, but I've raised that separately, at WebAssembly/esm-integration#41.)

[lukewagner]

Ah hah, I misunderstood the two 100% cases you were talking about. Yes, you can definitely mix and match inline and out-of-line modules like so:

(module
  (import "./out_of_line.wasm" (module $M1 ...))
  (module $M2 ...inline code...)
  (instance $m1 (instantiate $M1 ...))
  (instance $m2 (instantiate $M2 ...)))

Importantly, instantiate doesn't care whether $M1 and $M2 are in-line or out-of-line (only the module type matters) and so a bundler tool can mechanically switch between the two representations on a module-by-module basis.

[littledan]

Thanks for bearing with me and explaining. I don't have more to add or worry about, but I also feel like I still don't understand the implications. I'm closing this issue as there's nothing actionable at this point.