HTML Modules
justinfagnani opened this issue Β· 189 comments
Now that JavaScript modules are on the verge of widespread browser support, we should think about an HTML module system that plays well with it. HTML Imports doesn't have a mechanism for exporting symbols, nor can they be imported by JavaScript, however its loading behavior is quite compatible with JavaScript modules.
@dglazkov sketched out a proposal for HTML modules here: https://github.com/dglazkov/webcomponents/blob/html-modules/proposals/HTML-Imports-and-ES-Modules.md
The main points are that, using the JavaScript modules plumbing, you can import HTML.
Either in HTML:
<script type="module" url="foo.html">or JavaScript:
import * as foo from "foo.html";But within the scope of that sketch there are still several questions about the specifics of exporting symbols from HTML and importing to JavaScript.
@TakayoshiKochi made a polyfill (based on the <script type-module> polyfill by @matthewp) that implements one set of choices to the questions, and shows JS and HTML modules working together: https://github.com/TakayoshiKochi/script-type-module/tree/html-module-experiment
Thanks for starting the discussion, @justinfagnani!
For context for others, I've been involved with @TakayoshiKochi and others trying to firm up what HTML modules means before we present it widely. There's still a variety of possibilities. Here's my high-level overview:
Writing a custom element that authors can include using a single file is a bit tricky. In particular, how do you get your template HTML? (Which is often literally a <template>.)
- If you want to stay self-contained, and thus friendly to consumers, you're currently forced to inline your HTML into your JavaScript as a template string, then later use
innerHTML. This is costly; it requires first parsing it in JavaScript, then later as HTML, and using up extra memory for both representations. It requires all-at-once parsing, instead of streaming, and requires it to be done on the main thread (I think). And it defeats the preload scanner for any subresources. - If you want to "use the platform", you'll have to ask your consumers to include the
<template>in their index.html, with some well-known ID. Then all of the above disadvantages disappear, but consuming your library is very un-ergonomic.
At a high level, HTML modules wants to solve this problem. I think the case for it being a worthwhile problem would be stronger if we had data on the costs of the HTML-in-a-JS-string solution, but just by inspection it seems pretty bad. Still, I hope we can provide some data.
The next big question then is about developer ergonomics: what should the authoring and consumption experience for web components be?
- At one extreme, we could just allow importing of inert DocumentFragments via the JS module system. This solves the above problem in a minimal, simple fashion. But, it means you cannot collocate your HTML and JS into a single file. Most modern frameworks seem to aim for a developer experience of collation, from React's JSX, to Vue's .vue files, to Polymer's HTML imports usage, to Angular's HTML-in-a-JS-string approach.
- At another, we could work to add some opinionated semantics to imported HTML files that make them good as a "container" format for a web component. This is the approach @TakayoshiKochi has gone with, and I think it's a good illustration. It's much less of a primitive, and requires adding new unusual semantics (e.g. executing scripts outside the main document), but that might be OK.
Hope this helps set the stage!
Thanks for getting a public discussion going, @justinfagnani!
I see HTML Modules as being a good container format for achieving generic subresource dependency specification. They solve a need for platform-level colocated <template>, <style> and <script>/Module in a similar style to what I would get with something like Vue's .vue files or Polymer's existing HTML Imports. I'd love for them to sufficiently export a JS module - I found this to be a shortcoming of HTML Imports when I last used them.
Some questions:
-
What it is about HTML Modules that makes them specific to Web Components? If a third-party decides to just use HTML + CSS + JS in this container format, that should just work too right? I think we should cater for a few different styles of component to work well with HTML Modules.
-
Are HTML Modules primarily a DX convenience for containment? or do we also foresee trying to bake in a performance carrot (e.g preload scanner benefits) to the design?
-
How do we see HTML Modules fitting with respect to the broader Web Packaging efforts?
I also want to point out that HTML modules would not prescribe how component developers package their libraries, or even prescribe Web Components for that matter.
There are a number of styles that components could be written and/or packaged:
- Pure JS, using either template strings, DOM builder libraries, JSX, etc. (no HTML modules)
- Logic in JS, template/CSS in HTML:
import template from 'template.html';in the component's module. Maybe eventimport style from 'styles.css';is we include CSS modules. - Single-file, like Vue and Polymer: JavaScript, HTML and CSS all in one file.
These styles could all be compatible.
These styles could all be compatible.
That's a good point. Even if we go with an opinionated container format with new capabilities, like @TakayoshiKochi's prototype, it will likely be possible to opt-out of those new capabilities, e.g. by just making your main element a <template> so that everything inside it is inert.
It looks like @TakayoshiKochi's experiment doesn't create a new document for the imported module. Not sure if this is an intentional omission or not, as the @dglazkov sketch does mention there being a new document like in html imports. Here's an example module for those interested.
In the interest of listing problems and not solutions I would say some things I want (in addition to other things mentioned by people above) are:
- I want to be able to export JavaScript modules from an HTML module.
- I want to be able to query for elements within the HTML module.
- I want to be able to "reach in" to an HTML module some how from the outside and use stuff other than the HTML modules within. But maybe if (1) and (2) are satisfied that would mean that an HTML module could make the other parts (like the
<template>) accessible?
Thanks @justinfagnani for starting this thread! Thanks @domenic for the summary.
I was wondering if this is a good place as @addyosmani mentioned if HTML Modules may not be
specifically for web components. But as it is, this is good starting here in public:)
We're open to any questions / problems for HTML modules.
Feel free to pour your thoughts here.
FYI, here's slide deck I presented in BlinkOn back in February:
https://docs.google.com/presentation/d/1ksnC9Qr3c8RwbDyo1G8ZZSVOEfXpnfQsTHhR5ny9Wk4/edit
Re @matthewp (#645 (comment))
It looks like @TakayoshiKochi's experiment doesn't create a new document for the imported module.
Yes, it was intentional. Imported HTML is treated as DocumentFragment (as opposed to Document), so we don't have to bother about various semantics to be satisfied, although it yields some weirdness (e.g. why/how can <script> inside be executed?). I'm interested if you prefer Document (other than that was described in Dimitri's doc) to DocumentFragment.
If you're interested in history, here's record of Document vs DocumentFramgent for HTML Imports:
https://www.w3.org/Bugs/Public/show_bug.cgi?id=22305
We are interested in hearing requirements from framework authors, as HTML modules is not (should not be?) specific to web components definition (= container for custom elements and shadow DOM definition).
I admit that my experimental polyfill was biased towards HTML Imports + web components, and would be a good fit for Polymer or Vue use cases, but I would be more excited if the resulting HTML modules would be usable for more than these frameworks or libraries.
In my blink-dev thread, a developer from Scirra introduced me his very inspiring blog post:
https://www.scirra.com/blog/ashley/34/html-imports-are-the-best-web-component
The post is long, but I recommend reading it from top to bottom - my interpretation is that HTML Imports fit their product development requirement quite well, even though they do not use Shadow DOM and do use only little Custom Elements. That exemplifies the need for something that supports scalable development on web platform, and HTML Imports is one key primitive for it.
Here's my take on this. In today's world, developers are putting HTML inside their JS, not JS inside their HTML. The fact that we don't currently preload resources in a string literal, or we end up two string representations of HTML if you used string literal to include HTML in a JS file is a pure implementation detail. There is nothing preventing us from implementing an optimization if there was a well known symbol which was used to tag a HTML in a JS.
Furthermore, for various performance reasons, we really need to have a mechanism to define an in-file ES6 module; a mechanism which allows multiple modules to be defined within a single JS file. Once we have that, then it's very natural to define three modules: one with HTML, one with CSS, and one with JS in a single JS file.
So I'm not all convinced that we need to put JS into HTML as done in the HTML import. In fact, we'd likely oppose to such an approach on the basis that popular JS frameworks such as React have gone to the completely other direction of embedding HTML inside JS.
@rniwa I find this opinion confusing given your interest in template parts. Are you not interested in that feature either, now? Or are you saying you'd prefer to define your <template> (with parts) as a string literal in JS rather than in HTML. Why?
the basis that popular JS frameworks such as React have gone to the completely other direction of embedding HTML inside JS.
While not being a framework author nor a browser developer here, just an end-user, I'd like to say that, although it is a widely adopted approach, it still requires tooling and you'll not be able to serve vanilla HTML, CSS and JS unprocessed by stuff like Webpack and so on.
I'd also like to remind you that React was initially built with PHP where including HTML was once widely used. But is such a mixing really considered as a good practice by PHP developers now?
Furthermore, how shall HTML modules work in browsers with disabled JavaScript? If markup stays in HTML, I can imagine them still working somehow. But what if it is placed in string literals?
I wrote this blog post. I'm interested in participating in work for HTML modules, and I think our experience is valuable: we have published a commercial PWA using ~300 HTML imports.
As I touched on in the blog, I quite like the idea of being able to import a HTML file and get a Document, much like XHR for type "document". So this:
import * as doc from "./myimport.html"
would be similar to assigning doc the Document resulting from fetching myimport.html. You can then querySelector etc. on doc and use its DOM content in the JS module. (IMHO, this is far better than pasting chunks of markup in to strings in JS)
Then all you need to do is add a HTML tag that can propagate to further dependencies. I suggested changing
<link rel="import" href="import.html">
to be equivalent to
<script type="module">
import * as doc from "./myimport.html"
</script>
It could equally be <script type="module" src="myimport.html"></script> to do the same. This means further scripts/imports/modules are loaded and run, etc.
I have to say style application from imports is a useful feature for us - see my example of a dialog definition for an example. It's possible a framework could work around this (e.g. applyStylesFrom(doc)), but it seems a shame if the dialog definition example can't work natively.
If HTML modules covered that, I'd guess that largely covers everything we use imports for. In our case they're largely just the architectural scaffolding that brings together all the components of our web app, and as I tried to emphasise in my blog, they excel at that, particularly since it covers all three aspects of a web app (markup, style, script, at least until Google roll back style application π’).
I am strongly against trying to use script for any markup or style content. I know some frameworks are having success with that approach, and so we shouldn't do anything that makes that harder. But if you have 1000 lines of markup, surely nobody wants to paste that in to a script as a giant string? I strongly believe that there should at least be the provision for leaving markup in HTML files, and not orient the spec around doing what one framework does today. Remember frameworks come and go over the years. In our case, our PWA is notable for not using VDOM, has almost no use of markup-in-script, uses plain markup (no Angular-style directives), no templating (like {{this}} in markup), and no two-way data binding. I don't think any other framework on the web today takes this approach, but I believe our PWA proves that you can still build scalable, complex web apps that way. I do worry that people have framework-blindness where they think things should be done like the frameworks do, rather than thinking in terms of providing general-purpose capabilities that frameworks of different designs and approaches can make use of, even in the future when everything likely changes.
Anyway, here's a couple of extra ideas that may be useful to extend the utility of HTML modules as I described them. Perhaps it'd be useful to add a selector to pick out specific elements, e.g.:
import "#elem" as elem from "./myimport.html"
Now elem is the DOM element returned by matching the selector #elem on myimport.html. Not sure how to handle multiple results (e.g. using ".myclass" as a selector) - maybe import ".myclass" as [resultsArray] from ".myimport.html"?
Finally I don't know if it makes sense to export from a HTML module, but perhaps you could have something like:
import myElem from "./myimport.html"
where myimport.html has a special tag like
<export name="myElem">
<div>This div is an exported element!</div>
</export>
Perhaps that could also be reconfigured to re-export a JS module that the HTML module imports, e.g:
<export name="submodule">
<script type="module" src="a.js"></script>
</export>
...then allowing import submodule from "./myimport.html", which propagates the exports from a.js through the HTML import and in to another JS module. I feel like I'm shooting in the dark there though!
Furthermore, how shall HTML modules work in browsers with disabled JavaScript? If markup stays in HTML, I can imagine them still working somehow. But what if it is placed in string literals?
I can answer that, at least: they won't. HTML modules are part of the module system, which is entirely dependent on JavaScript.
Even HTML imports never worked in browsers without JavaScript. (The document would be "imported", but useless, since you need script to access imported documents.)
Even HTML imports never worked in browsers without JavaScript. (The document would be "imported", but useless, since you need script to access imported documents.)
Depending on future developments of course. Declarative custom elements (which we'll talk about at TPAC) presumably wouldn't need script enabled.
Declarative custom elements (which we'll talk about at TPAC) presumably wouldn't need script enabled.
Uh... let's just say I disagree with you very strongly on that, and we can leave that discussion for another time, instead of derailing the thread.
@rniwa I wouldn't want to encode only what has become most popular today for two reasons:
-
Because where we are now is largely a consequence of what capabilities the platform has had. JavaScript developed popular user-land module systems and loaders long before HTML Imports, and is now getting widespread native modules before HTML. Given the lack of support HTML Imports has, Polymer has had to carefully weigh it's options for working in browsers without polyfills. If Polymer had decided to support a pure-JS distribution that would have been be a pragmatic choice only and not an indication of what we'd really like to use.
-
The assertion (developers are putting HTML inside their JS, not JS inside their HTML) might not be so absolute. Polymer and Vue are current examples of libraries that support single-file HTML components with scripts. There would be much more if it were a native capability.
I'm not saying JS shouldn't also gain pre-loading and multi-module support, of course.
The only way I can imagine HTML modules being useful with script disabled is if they basically inserted their full DOM content at the place they were included. So to use HTML imports as an example, <link rel="import" href="import.html"> would be equivalent to copy-pasting the contents of import.html in place of that tag. I guess it would actually be nice to have a native way to have your header and footer sections defined in one place.
I get that some people are annoyed by the obnoxious things JS does on some pages. But if you're building a web app, it's simply inconceivable to develop it without JS. It would be like writing an Android app with Java disabled - can you still produce a useful app? The web is still an amazing resource for static documents which can work with JS disabled, but on the whole I think modules are aimed very much at the app side of development, where it would naturally pair with JS logic for an app. Still, I don't see any reason to exclude the HTML-only case. Maybe an include attribute, like <script type="module" src="import.html" include> , could specify to just paste in the DOM content there?
Perhaps we need to figure out what the declarative syntax for shadow DOM / custom elements will look like before we can figure out what HTML import should look like.
It's hard to talk about packaging a thing without knowing what the thing looks like.
For including HTML into HTML, probably you can imagine there is a <include src="..."> HTML tag that magically does what server-side include did on the client side. It would be nice to have.
However it is a separate problem or use case from HTML Modules - which wants to load HTML as a resource and do something on it with scripting.
Yes, I think the include case should be split out in to a separate discussion. Maybe a new issue should be filed for it?
@AshleyScirra if you want it to happen, whatwg/html would be more appropriate place to file an issue than here.
@rniwa declarative Shadow DOM / Custom Elements would be nice to have, but we don't have to be blocked on it, because assuming it's all written in HTML and no script then it is just a plain HTML. Although I can imagine that declarative syntax would not be such a simple thing because its semantics would involve hooks into HTML parser and DOM tree construction...
Furthermore, for various performance reasons, we really need to have a mechanism to define an in-file ES6 module; a mechanism which allows multiple modules to be defined within a single JS file. Once we have that, then it's very natural to define three modules: one with HTML, one with CSS, and one with JS in a single JS file.
Is this idea presented or discussed anywhere? This is very interesting and I would like to learn more about it.
So I'm not all convinced that we need to put JS into HTML as done in the HTML import. In fact, we'd likely oppose to such an approach on the basis that popular JS frameworks such as React have gone to the completely other direction of embedding HTML inside JS.
Real-world web development is constrained by actual implementations of browsers available and used today, and React may be the optimal thing to do modular development on the current web platform, but as we are expanding (relaxing?) the constraints of the platform, we don't have to be trapped in the local optima. (that said, I don't say we should ignore React's popularity)
So I'm not all convinced that we need to put JS into HTML as done in the HTML import. In fact, we'd likely oppose to such an approach on the basis that popular JS frameworks such as React have gone to the completely other direction of embedding HTML inside JS.
I would be very curious to hear opinions from some framework authors - e.g @sebmarkbage (React), @developit (Preact), @yyx990803 (Vue) and @IgorMinar (Angular) on the direction of embedding resources like styles and HTML inside JS. I acknowledge this is an increasingly popular pattern, however I'm curious if the availability of HTML Modules where colocated resources (including ES Modules) are a first-class citizen would actually be something they would find useful.
In particular for the final bundles that get generated from their tools/Webpack.
Vue, at least during the dev/iteration workflow appears to have shifted closer to a HTML Imports-type thing while preserving some of the options for authoring in JSX or styles-in-JS if you really want to. It still converts everything to JS during the build cycle.
Furthermore, for various performance reasons, we really need to have a mechanism to define an in-file ES6 module; a mechanism which allows multiple modules to be defined within a single JS file. Once we have that, then it's very natural to define three modules: one with HTML, one with CSS, and one with JS in a single JS file.
Do you see the need for a mechanism offering multiple ES modules in a single JS file being much different to a HTML module being the container enabling that? e.g
bundle.1.html
<script type="module">...</script>
<script type="module">...</script>
<script type="module">...</script>
declarative Shadow DOM / Custom Elements would be nice to have
I'd like to emphasise that a lack of declarative shadow DOM is a pretty massive complaint from the community and shouldn't be dismissed as a nice to have.
I'd like to emphasise that a lack of declarative shadow DOM is a pretty massive complaint from the community and shouldn't be dismissed as a nice to have.
I don't think it's a dismissal at all. In the context of HTML Modules, the question is whether or not declarative custom elements and shadow roots should block moving forward on modules.
π just making sure it's being heard is all.
@addyosmani I do think this general architecture is valuable. It can be used to import both somewhat static content but also for loading pieces of a server-rendered page out-of-order and inject dynamically. So, I'm all for it for frameworks that are HTML heavy and centric.
However, for React, and I believe this is a more general trend, we're increasingly looking for ways to avoid passing static, or server responses, as HTML as the serialization format. For these use cases we might use custom byte codes like Glimmer or some other format. There's a lot of value in a richer serialization format that can include more data to pass to JS, cheap branching and conditional logic, immediate access to references to nodes etc. I'm asking myself what value HTML, as the serialization format, offers to a JS library once it's already up and running. I see soft value like familiarity, and concrete value like browser specific caching formats. However, those have to be weighed against other things.
Even if you use what seems to happen is that you end up with two modes in the JS library to deal with nodes created from HTML and nodes created from another mechanism. That adds complexity and code.
Can you clarify
you end up with two modes in the JS library to deal with nodes created from HTML and nodes created from another mechanism. That adds complexity and code.
Node in HTML Modules would be standard Nodes in the rest of the DOM, there wouldn't be different types.
The types are the same but the mechanism for managing them (add/remove/update) is different. JS libraries like the ones mentioned above use DOM manipulation to do the updates. There's also often some richer data structures associated with each node too. The fact that this manipulation is very different from the initial load is what adds complexity in the library.
I'd like to emphasise that a lack of declarative shadow DOM is a pretty massive complaint from the community and shouldn't be dismissed as a nice to have.
For declarative Shadow DOM, see #71
Declarative Shadow DOM can be built on top of other primitives.
Unless there is a strong reason of having it as the platform's primitive, no one is likely to work on standardizing declarative Shadow DOM.
@hayatoito and @treshugart I think we should take it back to #71 or a new issue. Declarative shadow root is really useful for server-side rendering and declarative custom elements.
Furthermore, for various performance reasons, we really need to have a mechanism to define an in-file ES6 module; a mechanism which allows multiple modules to be defined within a single JS file. Once we have that, then it's very natural to define three modules: one with HTML, one with CSS, and one with JS in a single JS file.
Is this idea presented or discussed anywhere? This is very interesting and I would like to learn more about it.
The idea is to allow a declaration of multiple modules per file.
e.g.
import X from module { export const X = 1; }
import Y from module { export function add(x, y) { return x + y }; }Now suppose we had the capability to import a pure HTML file and a pure CSS file as follows. I'm inventing a new syntax import X from Y as Z where as Z defines a custom module importation behavior. Also note that HTMLStyleElement.prototype.sheet is readonly at the moment but I'm making it writable here to make it possible to set the style created in a script.
import template from 'my-template.html' as HTMLTemplateElement;
import styleSheet from 'my-style.css' as SheetStyle;
MyCustomElement extends HTMLElement {
constructor() {
const root = this.attachShadow({mode: 'closed'});
root.appendChild(document.importNode(template.content));
root.appendChild(document.createElement('style')).sheet = styleSheet;
}
}Combining these two features we would be able to do something like this:
import template from module `
<div>hello, world</div>
` as HTMLTemplateElement;
import styleSheet from module `
div { color: green; }
` as StyleSheet;
MyCustomElement extends HTMLElement {
constructor() {
const root = this.attachShadow({mode: 'closed'});
root.appendChild(document.importNode(template));
root.appendChild(document.createElement('style')).sheet = styleSheet;
}
}For the inlined case, we could simply introduce a template literal function like this:
const template = HTMLTemplateElement`
<div>hello, world</div>
`;
const styleSheet = StyleSheet`
div { color: green; }
`;
MyCustomElement extends HTMLElement {
constructor() {
const root = this.attachShadow({mode: 'closed'});
root.appendChild(document.importNode(template));
root.appendChild(document.createElement('style')).sheet = styleSheet;
}
}
import template from 'my-template.html' as HTMLTemplateElement;
@rniwa, that's an interesting idea. It reminds me of what webpack loaders do with their thing-loader! prefixes. This would also let the different import types recommended here exist simultaneously. Specifically, template / document fragment -style imports, which have almost new no semantics to define, could be implemented quickly and wouldn't prevent the document-style import from making it in once all the details are worked out:
import x from 'y.html' as Document;I think this syntax would be really useful if there was a way to hook into this and customize how raw data fetched by the loader were handled at runtime - i.e. a define a function that converts ArrayBuffers to something representing a module record. Maybe Symbol.importer? I think it would be necessary to import your importer [1] for the concept of 'referencing an import in an import statement' to make any sense though [2]:
import png from './my-png-importer.js';
import x from 'x.png' as png;
assert(x instanceof Image); // maybe this importer creates Image instances?
assert(png[Symbol.importer](someArrayBuffer).default instanceof Image); // whatever, it's a function.DocumentFragment[Symbol.importer], HTMLTemplateElement[Symbol.importer], Document[Symbol.importer], and others could all then be implemented independently once they're defined. Also, the actual module specifier is unaffected, so the loader can still statically determine the location of dependencies or continue to do whatever the loader spec folks want it to, assuming that's still a thing.
[1]: Or, it would have to be defined on the global scope before the module is executed, like HTMLTemplateElement and other browser globals are.
[2]: Yes, I would rather import importers than add another global registry. :)
import x from 'y.html' as Document;I think this syntax would be really useful if there was a way to hook into this and customize how raw data fetched by the loader were handled at runtime - i.e. a define a function that converts ArrayBuffers to something representing a module record. Maybe
Symbol.importer? I think it would be necessary to import your importer [1] for the concept of 'referencing an import in an import statement' to make any sense though [2]:
Right, we need TC39 to define some hooks to let the host language and authors to define a custom importation behavior. The way you defined Symbols.impoter is exactly I'd imagine this feature would be defined.
We could even add a sugar on top of it to improve the developer ergonomics further like letting the host language pick a custom importer automatically based on MIME type. e.g. "X.css" could automatically use StyleSheet's Symbol.importer but that's sort of nice-to-have.
Oh, maybe this would be even better if the function produced a promise for a module record. Then, your importer wouldn't have to be sync and could handle loading that resource's dependencies, if any, and would mirror import() nicely.
This:
assert(png[Symbol.importer](someArrayBuffer).default instanceof Image);would instead be this
png[Symbol.importer](someArrayBuffer).then(m => assert(m.default instanceof Image));which feels just like this
import(someUrl).then(m => /* do stuff */);I really like how general purpose this "import as" approach is. It looks like it covers a lot of cases for bringing in general-purpose sub-resources, even in varying formats:
import doc from "./x.html" as Document;
import img from "./x.png" as Image;
import imageBitmap from "./x.png" as ImageBitmap;
import imageData from "./x.png" as ImageData;
import buf from "./x.bin" as ArrayBuffer;
import blob from "./x.bin" as Blob;
import json from "./x.json" as JSON;
import str from "./x.txt" as String;
import audio from "./sfx.webm" as Audio;
import audioBuffer from "./sfx.webm" as AudioBuffer;With a few custom functions using Symbol.importer this looks like it could fairly straightforwardly be extended to some interesting new cases with the help of a small library:
import elem from "./x.html" as Selector("#elem");
import elemArr from "./x.html" as SelectorAll(".className");
// Automatically apply the CSS to the main document, as
// per original design of HTML imports
import style from "./x.css" as MainDocumentStyle;
// Custom implementations
import csv from "./x.csv" as CSVData;
import customerData from "./customerData.json" as CustomerData;
import canvas from "./initialImage.png" as Canvas;
import texture from "./texture.png" as WebGLTexture;I think this looks like a great feature, very flexible and general-purpose without too big a spec surface area.
My view on some other points:
- I don't see a need to use a string in these statements, I think you'd either import an external resource, or just use an inline string in script (as is already done today)
- I don't think there should be an
import ... as HTMLTemplateElement- it seems both a lot of work to define importers for each kind of element, and it's too limiting (how does it know which template element to return?). Something likeimport templateElem from "./x.html" as Selector("#idOfTemplateElem")seems better IMO, and more in line with how you'd use HTML imports today. - normal script imports don't return promises, so I don't think any of these imports should either. You can
import img from "./x.png" as Imageif waiting for that import suits you; otherwise you can always create a new Image and wait for its onload as you can today. No need to reinvent that wheel.
Actually, thinking of async imports, it could actually be supported with custom importers. Suppose import ... as X calls a function with this signature:
X[Symbol.importer] = function (url)
{
// return a promise when the import is done
}So as an example the JSON importer could be implemented like this:
JSON[Symbol.importer] = function (url)
{
return fetch(url)
.then(response => response.json());
}If there's some way to actually resolve with a promise, then you could implement imports which return a promise. The difficulty is, of course, that you can't resolve a promise with a promise, it will just wait for that promise to resolve and continue the chain. But if we try to work around that we could do something like:
AsyncJSON[Symbol.importer] = function (url)
{
// Return promise without waiting for it by putting it in
// an object property
return {
promise: fetch(url)
.then(response => response.json())
};
}Now you can do:
import jsonPromise from "./x.json" as AsyncJSON;
// wait for import to load
jsonPromise.then(json => useJsonData(json));If that works, there's no need to distinguish "sync" vs. async imports, it can be implemented either way via Symbol.importer.
This does a lot more than just HTML modules!
Actually, circling back to HTML modules, a custom importer could define:
import doc from "./module.html" as HTMLModule;The HTMLModule importer could be implemented in a library and do the following:
- fetch module.html as a document
- find
link[rel="stylesheet"]elements and apply them to the main document - find
script[type="module"]elements andimport()them - find a custom element e.g.
<html-module src="submodule.html">and recurse in to that for fetching further HTML modules
This moves almost the entire HTML imports functionality in to a library, rooted on the JavaScript module system. IMO this is the ideal situation - there's no need to specify HTML modules at all, it just becomes a framework built on top of generic imports.
- normal script imports don't return promises, so I don't think any of these imports should either. You can import img from "./x.png" as Image if waiting for that import suits you; otherwise you can always create a new Image and wait for its onload as you can today. No need to reinvent that wheel.
Actually, thinking of async imports, it could actually be supported with custom importers. Suppose import ... as X calls a function with this signature:
X[Symbol.importer] = function (url) { // return a promise when the import is done }
I wasn't clear but yes, this is exactly what I was going for there. Also, just to be more specific with regards to:
If there's some way to actually resolve with a promise, then you could implement imports which return a promise.
I didn't mean to imply every custom importer should produce a promise as its module record's the default export, but that the promise that the importer returns would itself resolve to the module record representing object. (Nicely covered by your examples for X and JSON.) So, I think we're on the same page. :)
Also, to correct the example I made earlier,
png[Symbol.importer](someArrayBuffer).then(m => assert(m.default instanceof Image));should really be
fetch(someUrl)
.then(response => response.arrayBuffer())
.then(png[Symbol.importer])
.then(m => assert(m.default instanceof Image));to get the same effect as
import(someUrl).then(m => /* do stuff */);while using a 'custom importer'.
@bicknellr any good reference to Symbol.importer? Besides what Google provides me. Thanks in advance π
Hey guys, currently I use webpack + plugin "raw loader" to do this. HTML Imports it's sucks. I wish I could do that native. I know you're arguing and just wanted to contribute as I do today. I do not want to use react, I do not want to use webpack, I want native Web Components.
Sorry for poor English.
import templateHTML from './report-date-picker.html';
export class DatePicker extends HTMLElement {
constructor() {
super();
const shadowRoot = this.attachShadow({ mode: 'open' });
const template = document.createElement('template');
template.innerHTML = templateHTML;
shadowRoot.appendChild(template.content.firstChild.content);
}
connectedCallback(){
}
}report-date-picker.html
<template>
<style>
:host {
}
</style>
</template>@alexgwolff π +1
@rniwa thanks for the detailed explanation and examples!
The original one:
import X from module { export const X = 1; }
import Y from module { export function add(x, y) { return x + y }; }looks like anonymous inline ES6 modules to me - so for performance, you can save two resource loads and their related overheads, right?
The original ES6 module syntax requires that all import statement should be statically parsable without evaluation - I'm not an expert of language scanner/parser, can the proposed syntax fulfill the requirement?
Also, as ES6 modules system starts running scripts when all imported resources are ready - which is different from HTML Imports which transitively parse and execute scripts. So for big chunk of data that is not needed at startup, maybe same mechanism is also needed to dynamic imports - though in that case fetch would work as well.
Yes, I think the include case should be split out in to a separate discussion. Maybe a new issue should be filed for it?
@AshleyScirra @milieuChris @rianby64 I filed whatwg/html#2791
@snuggs, no, I was just expanding on what was posted here. Maybe @rniwa has something?
The original ES6 module syntax requires that all import statement should be statically parsable without evaluation
@TakayoshiKochi, I'm no an expert there either but I bet the syntax here would work if the same restrictions imposed on export statements in modules currently also applied to their use in the body of an inline module. Also, I think they would need some way to reference one inline module from another without nesting. If they only supported trees then they wouldn't support everything that per-file modules do and wouldn't make for a useful build target. (Assuming that's their primary use case, maybe not?)
Here's not-yet complete summary of discussion here:
https://gist.github.com/TakayoshiKochi/fdc1e72d4fd0bbdaf61e7f23779e9139
I'll keep updating as we go, and please let me know if anything is missed, incorrect, etc.
To be clear, I'm in favour of generic imports (using Symbol.importer), since it appears to allow a variety of possible implementations of HTML Modules/imports in a JS library. That renders the whole question "what should HTML modules be?" moot, as it moves it in to a framework rather than being a specced web API.
I like the idea of importing non-JavaScript resources. Thankfully we require a MIME type for JavaScript modules so we shouldn't have trouble distinguishing the resources. I do think we probably want to define something around this as that would allow the browser to optimize parsing of these resources. If that was all delegated to a library you'd lose some of the benefits.
I think the other point @rniwa made is also a good one and was unfortunately somewhat hastily dismissed. That being able to write templates inside your JavaScript is something that keeps coming up and would be really good to have a good solution for, that also avoids common XSS pitfalls and such. whatwg/dom#150 discusses that in some more detail.
The number one problem I see with this is that this is going after multiple problems at once. While import * from './template.html' as DocumentFragment could be nice for JavaScript, at least in the context of Web Components it seems unnecessary. I'll quote @TakayoshiKochi here
What was wrong with HTML Imports?
HTML Imports solve the packaging problems of CE and allow Elements to be bundled individually or grouped together. See:
https://github.com/Nektro/custom-elements/blob/master/fl/fl-Button.html
https://github.com/Nektro/custom-elements/blob/master/fl/fl.html
The number one problem I see with this is that this is going after multiple problems at once.
Whileimport * from './template.htmlas DocumentFragment` could be nice for JavaScript, at least in the context of Web Components it seems unnecessary.
IMO, separation into multiple types of import behaviors enables a key benefit (for both web components and script in general). If we can avoid having to define the one true way to import HTML with import * from './template.html'; then we can avoid the inert DocumentFragments vs. templates vs. HTML-Import-like debate. This wouldn't preclude something similar to HTML Imports from being defined later and we also wouldn't have to block the ability to import HTML from script, as a whole, on that definition (assuming something like fragments or templates would be simpler to spec).
Definitely, both JS Imports and HTML Imports could work great together! Now that I've had a second thought though, some of the examples above seemed like they were more leaning towards a roundabout way to fetch but with import statements. In which case it seems like it would be more beneficial to simply add more options to the Response object.
I was under the impression a key part of HTML Modules would be to integrate them with the ES Modules system, so there's only one set of dependencies. For this reason having import doc from "./template.html" syntax seems like the best way to go about that. Then allowing the actual import behavior to be customisable with JavaScript allows for a wide range of capabilities, including building HTML import/module like behavior in a library.
This is all orthogonal to using HTML strings in script - it doesn't do anything to get in the way of that, nor does it help make that easier. However it's a big help if you want to keep your markup in HTML files, since it provides an elegant way to integrate other dependencies with your script.
Question for all: if you write import doc from "./template.html", template.html is...?
- just an HTML snippet that cannot have any sub dependencies in terms of module dependency tree
- it can have further dependency graph which may be a mix of html and JS modules
As far as I read in the discussions above, most people seem to assume 1, and importing JS script just consumes the HTML as an inert DOM tree. What I presented in the polyfill had capability to contain a JS module script in the imported HTML, which can import further modules, or export something to the importer.
The capability HTML Imports have had is that any import can have subimports, to form a dependency tree, and within an import, <script> is executed (though global, not scoped like ES6 module) in deterministic order. Also, during loading imports is processed, Custom Elements upgrades are done accordingly within imports (I wrote doc about it).
The current Polymer depends on this capabilities, to realize almost-declarative element definition (<dom-module>), which are so powerful.
I was imagining #2. But if we use the generic imports idea:
- HTML snippets can be imported via a built-in
import doc from "./template.html" as Document - Further dependencies can be imported via a library which defines HTMLModule as an object with a
Symbol.importermethod inimport doc from "./template.html" as HTMLModule. This would use the approach I described earlier, which would look through the fetched HTML for things like further script modules, and import those (presumably using the dynamic import feature). This covers propagation through the dependency graph via a library.
So the way I see it, the generic imports idea covers both.
@TakayoshiKochi @AshleyScirra as are you i'm big on real world implementation not theoretics. One pattern we've used as a lowest common denominator while the smoke clears is to assume whatever is imported is a bonafied Document (or HTMLModule depending on context). (currently the .import spec in our particular case polyfilled by webcomponentsjs). We then assume there is a "master template" within the import body of the document. This is the <template> used to "hydrate" each custom element found in the document of the same localName. This allows us to have the imported <template> contain subresources (i.e. <script>s, <img>s etc.) without triggering a network request. Yet also leaves open the option to achieve dependency resolution the way HTML Imports was initially intended to mitigate this ancient problem.
We also use the markup around the <template> to provide a lightweight method of embelishing the custom element with "self documentation". Therefore by navigating to the url of the "import" one can see how to use the import within their own master document. Then the module is reused depending on the occurrences within the DOM. (See working example here)
Now HOW the Document/HTMLModule is created/fetched is an implementation detail. The fact it's a not a string of text that needs to be parsed is the most important part IMHO. Less code gymnastics and better developer ergonomics.
I could have sworn somewhere in the spec for XMLHTTPRequest there is a hidden way to convert the response to an HTML document or access to a DomParser within implementation. Could be wrong.
Hope this helps as a valid use case!
The core feature for me here is getting a tree constructed from HTML by the browser, so either would be satisfying for me. However, if this is posed as a mutually exclusive choice, I would prefer the first option: "just an HTML snippet that cannot have any sub dependencies in terms of module dependency tree". It seems likely, to me, that the non-inert document-with-dependencies module type would take more time to define, more time to implement, and result in new or inconsistent behavior for elements in that module's document compared to the main document (example). I don't want to prevent 'HTML as a complete container format' from being implemented - there's clearly lots of value in it - but I also don't want the ability to import a pre-constructed tree by referencing HTML to be blocked on reaching consensus for the behavior of that container format. Although, given the amount of time HTML Imports have existed, the shape of this container might be clearer now than it was before?
With regards to writing templates directly in JS: if these templates are defined in strings, it doesn't seem very different when compared to the current state of the world, from a development perspective:
import template from module `
<div>hello, world</div>
` as HTMLTemplateElement;vs.
const template = document.createElement('template');
template.innerHTML = `<div>hello, world</div>`;But they could be a useful step towards modules as a build target, especially if this could be used with data that wasn't HTML. I suppose the value here depends on how adoption speed of HTTP/2 (how expensive it continues to be to split files) compares with the implementation of this feature.
However, if the idea there is instead to introduce a better, not-strings syntax for constructing trees in JS then that's a different story and I think that could be valuable on its own. It would be less related to the concept of generically customizable modules but would reduce some of the specific need for HTML modules.
but I also don't want the ability to import a pre-constructed tree by referencing HTML to be blocked on reaching consensus for the behavior of that container format.
Agreed @bicknellr. Been waiting for this my whole development career.
Unless I've misread, I think for @TakayoshiKochi 's first option, whatwg/html#2791 could fill that gap, no? and we could leave imports for the more sophisticated dependency-tree-resolving use cases.
I think includes or imports could use as an inert wrapper for any arbitrary markup including scripts, and non- markup should otherwise be treated as "live" in either the dumb inclusion scenario or the module resolution scenario.
It seems likely, to me, that the non-inert document-with-dependencies module type would take more time to define, more time to implement...
Call me a broken record, but: the beauty of generic imports is none of this has to be standarised. Generic imports allows this whole aspect to be written in a library. All that needs to be standardised is the "import as" mechanism.
WHATWG would still have to standardize the DOM / HTML related stuff (i.e. the import behaviors for objects defined there) but, yeah, TC39 would only need to care about what import X from Y as Z; means for arbitrary Z.
I've just closed all (~20) open issues with "html-imports" label in this webcomponents repository.
Some of them contained somewhat relevant discussion, and I tried to capture them by adding a link to the comment for each.
Thanks all for answering my question.
I am reading the feedback that generally loading HTML as a dumb resource, though available not as raw HTML text but as parsed DOM tree, would be enough for many use cases. It's understandable, though I'd like to argue that it will not cover some aspects of HTML Imports.
I agree that generic imports is an idea that is worth pursuing. So my current thinking is, that whether we should divide the current discussion into generic imports part and other parts. At least, for separating concerns, it would be beneficial for both discussions clearer.
Re #645 (comment)
for @AshleyScirra, as @bicknellr already mentioned, generic imports would still go through DOM / HTML / TC39 standardization process.
As @annevk already mentioned (#645 (comment)), ES6 module imports require MIME type to be specifically JavaScript (e.g. application/javascript etc.) today, otherwise rejected. But by allowing some other MIME types (e.g. text/html or image/png), without as Z, loaded content could be inferred from their MIME types.
For HTML, if it is just for fetching and parsing, the capability is limited, which could be good, but that's not very different from what you can do today with fetch (though dynamic, and unfortunately Response doesn't have .document() while xhr.response has).
Moving back to the old comment (#645 (comment))
This moves almost the entire HTML imports functionality in to a library, rooted on the JavaScript module system.
This may be true, in terms of functionality. But what about performance (including loading performance)?
To review what HTML Imports are good at, here's a list:
- Transitive dependency loader with de-duping (ES6 modules have, but resolves statically, not transitive)
- Leverages existing native parser(s) for JS/CSS/HTML (generic imports could have)
- Non-blocking (async loading & parsing, ordered evaluation) (ES6 modules are sync, single-threaded)
- Allows declarative development (HTML-first vs. script-first)
- Easy transition from existing HTML resources
Another thing is that HTML Imports could use preload scanner, which may improve loading performance of subresources. Though the quoted blog is somewhat old, and in the age of HTTP/2 and <link rel=preload> we want to reevaluate its usefulness.
Sure you can XHR a HTML document, but the point of this is to use the ES module dependency system instead of duplicating it with HTML import's own dependency system. Being able to import a Document has two benefits over fetch/XHR:
- it can have a shared dependency system with ES modules, which helps do things like deduplicate repeated dependencies
- it uses the same synchronous style as other imports, i.e. it's convenient that the script doesn't begin to execute until all imports have finished, which is nicer to work with than lots of promises (particularly if there is nothing to do until a subresource like a document or image has loaded).
But what about performance (including loading performance)?
I was previously concerned about this, but it doesn't actually seem to be that big a problem, at least in our case. We recently switched over from native HTML imports to a polyfill based on XHR, and we could not measure any performance difference, and none of our users noticed anything different. I think this is because in our case we have ~300 imports which are generally all pretty small, so there's not much advantage to preload scanning. Also I suspect the network layer gets saturated pretty early on in loading and loading performance depends mainly on roundtrips (which is generally log(number of imports) assuming a reasonably balanced dependency tree).
Even if other cases show a significant performance difference, I think being able to move features in to a library is such a huge advantage that it would be better to do that and work out ways to fix any performance issues later on. This is exactly what the extensible web manifesto is about: build low-level capabilities, and then you don't have to spec or implement more advanced features on top of that, they can be left in a library. I'm sure there's plenty that can be done to optimise a library-based approach - for example if preload scanning is really important, surely there can be a special "fetch document with preload scanning enabled" mode added which the library can also make use of. It's even possible, looking further ahead, that someone could have a streams-based WebAssembly HTML parser and even do all of that preload scanning in the library.
Since libraries can iterate quickly, it's even possible they could be faster than a built-in. I'm not sure what HTML imports says about the order of sequencing loading stylesheets and script, but our polyfill assumes no scripts call getComputedStyle on startup and allows all style to load in parallel to script without waiting for them to finish. That's the kind of "cheating" optimisation you can experiment with, or enable as an option, in a library, since you can deviate from the strictness of the spec in the name of performance.
I'm convinced generic imports are a good idea anyway, so I think we should go ahead with that, and see how far we can get implementing HTML Modules on top of that. I'm willing to write the library that does it (we'll probably use it ourselves). It may be that we can actually make an optimal and production-grade HTML modules implementation on top of generic imports, in which case we saved a lot of spec and implementation work!
ES6 module imports require MIME type to be specifically JavaScript (e.g. application/javascript etc.) - @TakayoshiKochi
Is this inclusive of application/ecmascript? Although considered the official ECMAScript MIME type in rfc4392 often servers respond with OCTET streams and/or often goes unrecognized by browser and downloads similar to a content disposition. I know this is a corner case but curious of any caveats people like myself who utilize application/ecmascript MIME types in conjunction with .es extension within certain HTTP responses based on Content Negotiation.
If modules would require explicit application/ecmascript would like to know sooner than later so can adjust accordingly. Thanks in advance! π
/cc @annevk
The MIME types allowed for JavaScript modules are listed here: https://html.spec.whatwg.org/#javascript-mime-type (application/ecmascript is one of them).
application/ecmascript is one of them
ZOMG π π π thank you so much for this reference. I've mentioned about application/ecmascript and had actually gotten shunned before in the whatwg/dom or html issue tracker for that not being a valid mime type from a commenter. As if the RFC4392 wasn't enough. This was my missing link i've been looking for for ages. Being on the whatwg spec is irrefutable. Thanks @annevk
It is an obsolete MIME type though. See https://html.spec.whatwg.org/multipage/indices.html#mime-types-2. text/javascript is canonical; application/ecmascript is an obsolete synonym (similar to text/jscript or text/livescript).
In practice using obsolete types has no effect on any software. (At least, software that conforms to the HTML Standard and treats them all identical.) It just makes it harder for other developers to understand your setup. The purpose of picking one canonical type (text/javascript) is so that everyone can converge on that; i.e., it's a social purpose, not a technical one.
@domenic Keeping in mind i know we aren't discussing type attribute on <script> as that is inconsequential (for the most part) and should be left off. That said, I thought text/javascript was considered problematic in cases? Everywhere i've looked on the internet head nods towards application/javascript and actually states text/javascript to be considered obsolete moving forward since 2011. Don't know how authoritative you deem Dr. Axel but most of what i know about ES6 are from his blog posts. Also speaking of which the link you sent you mentioned "obsolete" however application/javascript has the exact same notation. Did I miss something? Unless you meant both are obsolete. In that case thoughts on these recommendations because mostly all CDNs state similar conventions. Should I consider RFC "wrong"? Feels a little strange doing that but understand if spec recs. nod towards a different direction.
Thanks in advance. π
/cc @brandondees
Yep, both application/javascript and /ecmascript are obsolete. text/javascript is canonical.
If others say something different, they are not following the spec. Which is fine, as I pointed out above; it only creates confusion, not any real harm.
@snuggs yeah, HTML replaces the RFC. I guess we should make that official at some point through the registry...
Average-joe-boots-on-the-ground-the-meek-shall-inherit-the-earth developer here :-). Apologies in advance for any lacking nuance or inaccuracies.
The picture I have in my head is a recursive yin yang symbol. Html and JavaScript have a symbiotic complementary relationship.
Today, Html can import JavaScript (external and inline), which can (now) continue to import JavaScript recursively. It can import styles natively. But, unfortunately, HTML (without imports or client-side includes, or iframes) can't import common html natively. I think this is a serious gap, especially with the advent of HTTP/2.
Also today, JavaScript can't import HTML or css files. I see benefits from all combinations working -- recursively, external and inline).
rniwa's proposal:
import template from 'my-template.html' as HTMLTemplateElement;
import styleSheet from 'my-style.css' as SheetStyle;or
import template from module `
<div>hello, world</div>
` as HTMLTemplateElement;
import styleSheet from module `
div { color: green; }
` as StyleSheet;seems brilliant on a number of fronts -- familiar, natural syntax, ease of polyfilling (?) and the ability, I presume, for parsers to optimize, and tools to colorize / autocomplete, etc, in a standardized, non proprietary way.
Perhaps there is some pitfall in doing this, but it would be fantastic if the template literals could evaluate constants, if that can be done without sacrificing optimizations / tooling:
const commonMarkup = `
<inner-stuff>
hello, world
</inner-stuff>
`
import template_1 from module `
<div>${commonMarkup}</div>
` as HTMLTemplateElement;
import template_2 from module `
<span>${commonMarkup}</span>
` as HTMLTemplateElement;This would allow large blocks of common html to be shared, without the ceremony of polluting the precious namespace of custom element tags, just for snippets of html. This would greatly reduce the payload (and maintenance).
The ability to have HTML files to be able to recursively import other html is also quite useful, for similar reasons, including activating inline and external JavaScript as well as style dependencies. This can be done as a custom element, I'm sure (essentially rewrite HTML import as a custom element), but I assume the browser would be faster if it implemented this natively?
Update:
I guess one could argue that once you have the ability to import html via JavaScript (presumably including activating JavaScript and Style dependencies?), the imported HTML could then recursively import other HTML via an inline script tag.
<script>
import template from 'my-template.html' as HTMLTemplateElement;
</script>Is that the thinking?
Would this be done synchronously? Would you need to use the dynamic import() to do it asynchronously?
familiar, natural syntax, ease of polyfilling (?) and the ability, I presume, for parsers to optimize, and tools to colorize / autocomplete, etc, in a standardized, non proprietary way
I don't think it's any of those things π and it certainly feels a lot less natural than
const template1 = new HTMLTemplateElement();
template1.innerHTML = ` ... `;
// Hypothetical
const styleSheet = new CSSStyleSheet(` ... `);Both are familiar and easy to polyfill.
Would this be done synchronously? Would you need to use the dynamic import() to do it asynchronously?
Both are asynchronous but static imports are done before the rest of the code is executed.
Text formats are easy to inline, but the nice thing about the generic imports proposal is it easily covers very large content, and also covers non-text formats which are not so convenient to inline (e.g. images, audio).
I'm interested in taking the generic imports idea further - what's the next steps to take to advance that?
Good points, and in retrospect, I would very much hope that the external HTML import from JavaScript, at least, could do more than just import a single innert HTMLTemplate, but could, with a single reference, import multiple html templates contained in the same file at once, and activate recursively importing other references, the way HTMLImports does from HTML documents.
@bahrus - I described exactly that approach in my earlier comment.
Please, consider also the case when a HTML import holds a Worker. It should be resolved against it's url, as this happens with scripts, images and so on...
Yeah, your proposal seems to be a lot more appealing, when it comes to external references.
But I do like the idea that there can also be some official way of inlining html templates, and style templates (and maybe other resources) in a way that the browser will optimize around, and which tooling can work with. With support for constants. Colating resources, in other words. If all these things happened I would be a happy camper.
To the specifics of your proposal, I would hope that there would be a way to apply the stylesheet to the ShadowDom context from which the reference was made, but I can also see the benefits of applying global styles too.
I'm also wondering if your proposal could be extended to support dynamic import() of HTML . Custom Elements already have a way to know when they are finished loading (customElements.whenDefined) but if all you want to do is embed a template html fragment into other html, knowing when the reference is ready would be quite useful, without blocking other pieces of the page.
But I do like the idea that there can also be some official way of inlining html templates, and style templates (and maybe other resources) in a way that the browser will optimize around, and which tooling can work with.
For inlining HTML, there are a lot of us who've come around to the idea of using something like JS template literals because of the fact that one of the main concerns with templates is intermixing values from script in the template output.
What I think would really help here is some kind of support for parsing HTML contained in template literals with a special tag, and producing an HTML template:
@html`<div>${name}</div>`(I used @html only because the parser would need some way to recognize the tag that's not an identifier that could be user-defined, and as of yet I don't think decorators are allowed in that position, some other sigil could be used).
There have been previous proposals around parsing inline HTML, like E4H, but they preceded the <template> element and missed out on the importance of templates and in-place updating of existing DOM, over the creation of new DOM.
I've implemented a template system based on this idea, plus template expression syntax discussed here: https://www.npmjs.com/package/lit-html
Hi, I updated the summary of this thread + some backgrounds.
Here's executive summary:
So far, most people seem to be interested in HTML modules (or whatever it will be called) as a way to construct an inert DOM tree from HTML in JavaScript. It is not limited for the use case of Web Components, but could be useful for various applications.
It comes in two flavors:
a) Extend ES6 module syntax: e.g. import x from './foo.html' as Document;.
b) Use ES6 template literal to instantiate from inlined HTML string.
@rniwa's #645 (comment), is the most informative comment of both. The two could be separate things, but rendering engine's implementation can share the internal pipeline (network stream or string -> HTML parser -> DOM tree) for both (the following discussion around Symbol.importer after the comment above is an idea how to do so).
@justinfagnani's #645 (comment) has implementation of b) + whatwg/html#2254.
Of course details should be fleshed out, but if there is no significant opposition against this direction, we can move forward.
And in case of Blink, if these can cover use cases of HTML Imports, we have no worry about deprecating HTML Imports and new primitives can polyfill HTML Imports if needed.
b) Use ES6 template literal to instantiate from inlined HTML string.
I don't understand why HTML modules have to be involved with this. Why is this necessary:
import template from module `
<div>hello, world</div>
` as HTMLTemplateElement;when you can already do this?:
const template = HTMLTemplateElement`
<div>hello, world</div>
`;If you want to inline a import template from 'my-template.html' as HTMLTemplateElement; statement as a build step, surely it's straightforward to transform that from the import to the const declaration?
There's no equivalent for inlining a script import either, which I would guess is because the whole intent of the import system is to bring in external dependencies.
AFAICT the inline case is already well supported by template strings. What benefit is there to reinvent this wheel with imports?
You could actually cover case b) using only case a) if you can provide a computed URL to import from. For example:
import x from inlineCss`.rule {}` as StyleSheet;This could simply create a blob URL to the inline string, and request that. Then we only have to implement generic imports with a URL, and get the inline case for free.
Putting inlining module aside, asking why import ... from "./foo.html"; is necessary is equivalent to ask why you need import ... form "./xyz.js";? You can surely inline xyz.js into your JavaScript code, with some namespace wrapping.
My understanding is that we use rendering engine's HTML parser + DOM construction code for importing HTML file, which can run in parallel to JavaScript parsing, and it gives opportunity for rendering engine to optimize the whole loading tasks.
I believe in your second example, HTMLTemplateElement function could be implemented in native code, and even JavaScript parser can pass the string to native HTML parsing + DOM construction code which may run in the different thread off the main thread, if the string doesn't contain any interpolation.
I don't think your 3rd example doesn't work, as import statement has to be statically resolved before evaluating the actual JavaScript code. Statically analyzable import statement is an absolute requirement for ES2015 modules, if I understand correctly.
So realizing as TypeName as an extension of import statement has a challenge that it has to be statically analyzable. Currently, ES2015 modules depends on MIME type checking if the imported file is a JavaScript module file, the whole module tree fails if any of them is not JavaScript. My thinking is we allow some non-JavaScript MIME types as a starter (e.g. text/html, or text/css) that can be imported, and grow types as needed.
To be clear, I was asking why the inline HTML case was necessary. I am perfectly happy with importing from a URL.
It's a good point that import statements need to be statically analyzable. Perhaps "import as" with a Symbol.importer could be used with dynamic import() only?
To be clear, I was asking why the inline HTML case was necessary. I am perfectly happy with importing from a URL.
Ah, okay.
It's a good point that import statements need to be statically analyzable. Perhaps "import as" with a Symbol.importer could be used with dynamic import() only?
As @rniwa wrote in #645 (comment) , MIME type can be used to infer the object type without as X, I think for some known MIME types we can make them work. But I have no idea how custom importer can work for static imports. @rniwa might have an idea.
I think the as part is useful to include still, even if limited to built-in types:
- it might actually help static analysis, since a static analyser might not know what the MIME type would be in advance
- it makes the intent explicit, which helps you understand the code
- it verifies the MIME type is the expected one, i.e. that you get an error if the MIME type accidentally changes, rather than silently switching the type of import.
I can see custom imports being contentious for static imports since they are inherently dynamic. Perhaps if dynamic import() handles custom imports, then you can at least do something like this:
// normal static imports
import * from '/foo.js';
import doc from '/foo.html' as Document; // built-in extension
// dynamic imports, library implementations with Symbol.importer
Promise.all([
import('/module1.html', HTMLModule),
import('/module1.html', HTMLModule)
])
.then(imports =>
{
// ...
});Something like a top-level await in modules would make this case look nicer, but a quick shows that is contentious:
// normal static imports
import * from '/foo.js';
import doc from '/foo.html' as Document; // built-in extension
// dynamic imports, library implementations with Symbol.importer
const module1 = await import('/module1.html', HTMLModule);
const module2 = await import('/module2.html', HTMLModule);
// ...As the other article points out, this appears to be inherently slower. Since static imports can be statically analysed, the browser can start fetching them all simultaneously, whereas dynamic imports end up doing a sequential set of requests as each one finishes loading. Maybe that means we should go back to using static import for custom imports so the browser can at least statically analyse all the URLs and start prefetching them... would we be able to have this?
// enables importing as HTMLModule, using library implementation in lib.js
import HTMLModule from '/lib.js'
import doc from '/module.html' as HTMLModule; // custom importBrowsers don't guess how to parse JavaScript based on MIME type or extension. They require type=module for ES modules, for instance. I think it'd be a bad idea to do it here.
- it might actually help static analysis, since a static analyser might not know what the MIME type would be in advance
- it makes the intent explicit, which helps you understand the code
- it verifies the MIME type is the expected one, i.e. that you get an error if the MIME type accidentally changes, rather than silently switching the type of import.
I think these are all valid. But I still see it's hard to be generic to every possible JavaScript type (core JavaScript ones + browser-implemented ones) or every MIME type, so we'd start from very limited set of types, probably.
Imagine:
import doc from './foo.html' as DocumentFragment; // okay, convert text/html to DocumentFragment
import cs from './foo.css' as CSSStylesheet; // okay, text/css to CSSStyleSheet
import json from './foo.json' as Object; // maybe okay...
import array from './foo-array.txt' as Array; // maybe technically possible, but...
import png from './foo.png' as Image; // maybe...
import data from './foo.xls' as Blob; // ???
import fromdata from './response.txt' as FormData; // ???I see text/html or text/css would be useful and they have use cases. To implement, if we support N types, we would have to implement N paths to hook from JS module loader to parse and feed the decoded data into JS world.
Re dynamic imports: sometimes you would only have to import bootstrap code and other resources could be lazily loaded, in that case custom importer can be in the bootstrap code and be useful for processing lazily loaded contents.
// enables importing as HTMLModule, using library implementation in lib.js
import HTMLModule from '/lib.js'
import doc from '/module.html' as HTMLModule; // custom import
In the constraints of how today's ES2015 modules load, this seems difficult.
IIRC in the past loader spec tried to have some filtering, though it seems it was removed.
Somewhat relevant issue there:
whatwg/loader#87
The spec seems to have got stuck for long time, so we might not expect it to go anywhere.
Instead, let's make progress here.
Browsers don't guess how to parse JavaScript based on MIME type or extension. They require type=module for ES modules, for instance.
Browsers do check MIME type before parsing JavaScript modules and rejects if they don't have valid JavaScript MIME types (see some comments above). I tend to agree e.g. <script type="module" src="./foo.html"> doesn't make sense (assuming we treat HTML module as a leaf node without any script execution and subresource loading), but other than that, what and why do you think a bad idea?
You're right about browsers checking the MIME type. When I say "guess how to parse" I mean only that the script and module forms of JavaScript do not require different MIME types.
Rejecting an import when the MIME type is incorrect is reasonable. I only flagged the inferring of import type based on file extension because it is different from the script tag example. I'm not familiar enough with any arguments for why script was handled that way, but I thought it was worth noting.
Thanks for the clarification! We do not do anything depending on file extension so far, and I don't think anyone proposed doing anything different than JS modules depending on extension in this thread, so I believe we are on the same page.
There was a dispute between ES modules vs CommonJS people about the interpretation of extensions, but as far as I understand it was settled not to honor file extension.
Thinking about this more, I don't think generic imports is an interesting feature unless it is dynamic. I don't see much value in import doc from './foo.html' as DocumentFragment if all it does is basically a fetch with type document. Why not just fetch it normally? The whole value of the feature comes from being able to run a JS hook and do things like implement HTML modules in a library.
Maybe there's another way we could approach this? How about adding a JS hook that is called for every import statement?
Yes, realizing generic imports in static import statement has lots of challenges to clear.
But still, plain import doc from './foo.html' as DocumentFragment would have values for a way to declare loading dependency to non-JS resources, and gives opportunity for rendering engine or HTTP server for optimizations. Then, HTML resource will be a simple dumb leaf node, and it is no longer "HTML modules", as it cannot have any dependency without any help of JS code to do processing on it.
So one view of HTML Imports vs HTML modules (the simplified one that only leaf HTML is imported) is that how you see building an web application by defining components and their dependencies in HTML-centric way (HTML Imports) or JS-centric (all in JS import) way.
Probably doing plain HTML import is the first step, and then how we hook the loaded resource would be the next step. We don't have to do it all at once, or all in one feature.
For hook ideas, loader spec already discussed in the past, so before doing it all over again, it's recommended to review what was discussed there.
whatwg/loader#62
whatwg/loader#147
If this spec is going to be dependent on loader specs it's doomed to fail. I hope we can continue forward with making HTML a first class type the way that JavaScript is, without waiting on loader. The same way that we were able to get script[type=module] and import() while leaving the door open for a loader spec in the future.
Browsers do check MIME type before parsing JavaScript modules and rejects if they don't have valid JavaScript MIME types @TakayoshiKochi @tilgovi
Are we sure about that? application/octet-stream (I don't think a valid mime) does work. Unless we are discussing what we want. Not what we already have.
@snuggs Yes, only Edge is wrong in this regard, all the other browsers will only execute a module if the mime type is one of the JavaScript variety. This is spec-ed here specifically part 7.
Is webcomponents works only for JavascriptοΌ if without JavascriptοΌI only want to create an contact card with avatar as components, but without javascript skillοΌ
I likes javascript only do something as javascript do, Itβs a promote for web development, not the only way, web needs be can works without any script tag.
<!-- contact-card/contact-card.html -->
<element name="contact-card">
<template>
<style>
:host { }
:host(sizing="cover") { }
:host(sizing="contain") { }
</style>
<img src="${avatar}" />
</template>
</element>
<!-- index.html -->
<link rel="import" href="contact-card/contact-card.html" />
<contact-card sizing="cover" avatar="avatar.png"></contact-card>
<contact-card sizing="contain" avatar="avatar.png"></contact-card>in javascript, we has fetch api already, I think allow javascript import html file will likes this:
import * as myModule from "my-module";
import doc from '/foo.html' as ??;
/* how to import ClassA from foo.html? */
import {ClassA} from '/foo.html' as ???;
/*
Is foo.html a Document or DocumentFragment?
import ClassA from document or fragment ?
*/
/* and could we import a stylesheet from foo.html? */
import {style1, style2} from '/foo.html' as ???;
/*
<link rel="stylesheet" export="style1"/>?
or
<export name="style1">
<link rel="stylesheet" export="style1"/>?
</export>
*/
/* if we had more than one template in foo.html */
//....imported html may contains many things such as templates, styles, scripts. the best way to access them also by javascript api, not only syntax support.
such as an component
<template>
<style>
</style>
<h1>Content</h1>
</template>
<script>
class TheComponent1 { }
class TheComponent2 { }
export TheComponent1;
export TheComponent2;
</script>The better way will be template move to filename.html script move to filename.js. or some one likes this:
import {TheComponent1 as Component1, TheComponent2 as Component2} from 'template.html' as DocumentFragment;
// or
import doc from 'template.html' as DocumentFragment;
let Component1 = doc.TheComponent1;
let Component2 = doc.TheComponent2;So I think html import is better than esmodule for webcomponents now.
<link rel="import" href="custom-element.html"/>don't forget in javascript, we have the fetch api, there just needs a __dirname or keep current file path for any esmodule to load some static content.
import {Element} from '@polymer/path/to/polymer-element'
class CustomElement extends Element {
}
export default CustomElement;
fetch('template.html', myInit).then(function (res) {
return response.text();
}).then(function (text) {
someTemplateHolder.add('custom-element', text);
customElements.define('custom-element', CustomElement);
}).catch(function(error) {
console.log(error);
});
// 0r
customElements.define('custom-element', CustomElement, {
template: 'template.html'
});
// Or just a decorate
@template('template.html')
class CustomElement extends Element { }
customElements.define('custom-element', CustomElement);
// Or
class CustomElement extends Element {
static get template() {
return 'template.html';
}
}There a lot way to do in javascript. so, import a html file in javascript is not required. Javascript can do this already. But the important is HTMLImport works without javascript.
I've written more about how generic imports could work and got a live demo with real running code at this repo: https://github.com/AshleyScirra/import-as-and-html-modules
It includes a working prototype of a library-based HTML Modules implementation.
This only needs a small API surface, the code is all pretty simple, and the feature is very flexible, so hopefully it's something compelling to consider.
(back from vacation, catching up. sorry for the delay)
@matthewp I didn't mean to depend on loader spec, but rather we don't want to repeat the same arguments which already happened in the loader spec, and we could learn from something from there.
@AshleyScirra thanks for creating the demo! At a glance it looks promising, will take a closer look.