tc39/proposal-asset-references

Better static resolution for dynamic/partial specifiers

jamiebuilds opened this issue · 7 comments

jamiebuilds commented

Writing imports like this are super common in the ecosystem:

import("./translations/" + name)

Tools like Webpack support this by turning the expression into a regex matcher like:

/\.\/translations\/.*

Which then pulls in every single file inside the ./translations directory (including any sub-directories) into the JavaScript bundle.

I've seen cases where people have accidentally added 10s of MBs of code to their JavaScript bundles by accident not understanding this behavior.

Not to mention all the complexities of having to support statically analyzing syntax like this. It's just outright no supported by many tools, and the ones that do support it have subtlety different behaviors.

In the past I suggested having something like:

let translationImporter = import.meta.glob("./translations/*.json")
let translation = await import(translationImporter("./translations/" + lang + ".json"))

Which would get desugared to:

function _import_meta_glob(glob, regex) {
  return url => {
    if (regex.test(url)) {
      return url;
    } else {
      throw new Error(`${url} does not match ${glob}`);
    }
  };
}

let translationImporter = _import_meta_glob("./translations/*.json", /^\.\/translations\/([^/]*)\.json$/);
let translation = await import(translationImporter("./translations/" + lang + ".json"))

Obviously globs are not something that is part of the Web or JavaScript at all (although they are the best solution I know of to this problem).

But if globs are totally unacceptable, maybe we can try giving some sort of better primitive for this?

bmeck commented

@jamiebuilds I'm not entirely sure if this is needed on the first iteration. I know I would like to be able to setup relative locations of assets dynamically. Globs get into territory like url parsing potentially with things like ./translations/*.json?foo presumably wanting to not allow ? in the * portion.

A tool could compile these glob patterns into a series of individual patterns potentially though.

asset translations from './translations/*.json';
fetch(translations['en']);

becomes

let translations = {};
asset en from './translations/en.json';
translations.en = en;
asset zh from './translations/zh.json';
translations.zh = en;

fetch(translations['en']);

I think the ability to have multiple paths makes it a bit more plumbing than just a simple reference type, which is part of why I want to put it off.

If we could support globs we should probably figure out if that list of internal mappings is immutable/static or dynamic. Using a function like you show above makes me think you want it to be dynamic, but I might be wrong.

jamiebuilds commented

The reason I think this is important to consider is that this is by far the more problematic scenario in the ecosystem, and the design of this proposal should lend itself to being extended for this purpose.

I'm not suggesting that globs are the perfect solution, but they are much better than regex (regexes against URLs are awful to write)

I don't think creating a key map is necessarily the best thing to do either, but if they were to be used, I'd give the fully qualified specifier:

asset translations from './translations/*.json';
fetch(translations['./translations/en.json']);
jamiebuilds commented

If we could support globs we should probably figure out if that list of internal mappings is immutable/static or dynamic. Using a function like you show above makes me think you want it to be dynamic, but I might be wrong.

I used a function because I don't think it needs to do anything more than asset the path you pass through matches the original glob/specifier

Another reason we wouldn't want to use a map of matched paths is that there would be no efficient way to discover them all at runtime

bmeck commented

@jamiebuilds one of the points of this proposal is to allow eager fetching/ahead of time tooling like is done with bundlers. How could we do that if we don't make a map of possible things to load?

Another reason we wouldn't want to use a map of matched paths is that there would be no efficient way to discover them all at runtime

I'm not sure I understand this test function. If you just pass in specifiers, you wouldn't need to iterate and discover them all. Wouldn't the same be true for mappings? If we only expose a query interface to a map that returns true or false if it exists in the map it would also not need to do runtime discovery.

jamiebuilds commented

How could we do that if we don't make a map of possible things to load?

It would be easy for a bundler to translate asset t from './t/*' into an object, that could then be accessed. But the browser wouldn't be able to do the same thing if asset .. from .. was not preprocessed.

one of the points of this proposal is to allow eager fetching/ahead of time

I know, but I'm more interested in these motivations for this particular issue:

I'm not sure I understand this test function.

The only purpose of the test function is to have early errors at runtime when requesting unexpected paths. Otherwise the runtime doesn't necessarily align with what we expect statically.

I would like to be able to ban code from specifying dependencies in ways that don't have both static and runtime guarantees that

// Good: Will throw error at runtime if runtime string doesn't match glob
asset translations from './translations/*.json';
fetch(translations(`./translations/${lang}.json`));

// Bad: Could silently not match glob
asset translations from './translations/*.json';
fetch(`./translations/${lang}.json`);

// Bad: Could silently not match glob
fetch(`./translations/${lang}.json`);
jamiebuilds commented

A good alternative to globs would be to use parameters:

asset translations from './translations/$0/$1.json';

await import(translations('en', 'dashboard'))

Edit: Although, you'd still need to decide what is valid for matching against that...

jamiebuilds commented

Another option being standardized elsewhere is https://github.com/WICG/urlpattern

asset translations from "./translations/:lang/:category.json"

await import(translations(`./translations/${lang}/${category}.json`))