Dependency post-processing?
jods4 opened this issue · 9 comments
From @niieani
Every dependency inclusion should have a post-processing step just before adding the dependency, one looks for centralized bundle configuration and depending on that makes the require lazy (or not), and places it into the correct chunk. The reason is that we might want to be able to use centralized configuration, which overrides the configuration provided via PLATFORM.moduleName(name, { configuration }) (useful for various reasons, including testing)
I am not sure I understand the use-case perfectly.
My plugins work as any built-in webpack plugins would, so if for example you want to override the chunking, that would be an external concern handled by another plugin (e.g. the built-in CommonChunkPlugin does that)?
The reason is that even if you define a chunk in the code (i.e. PLATFORM.moduleName(name, { chunkName: 'some-chunk', lazy: true })), there will be situations you in which you want to override that decision and force the module to NOT load lazily (maybe for unit-testing), or to do the opposite (when the PLATFORM.moduleName happens in an external module we have no control over -- in such a case we'd want to chunk it or make it lazy).
I have to say I'm not 100% sure I get it. Do you have an example somewhere? Maybe one of those unit tests?
there will be situations you in which you want to override that decision and force the module to NOT load lazily (maybe for unit-testing),
Well... modules designated by moduleName are meant to be loaded by Aurelia, whose loader is always async. You can't change that.
What you can change is whether the file containing your module will be loaded on demand, or not. I am really not sure why that matters as there is no observable difference from user code, but this is something that in webpack-land you would do during the chunk optimization phases. If you want to, you can easily write a plugin that collapses all chunks into a single file (and there are plugins already that can merge/split chunks aggressively based on code size, etc. This is untested but my plugins should work fine along those).
So I see chunk management as rather orthogonal and doable in a separate plugin/concern.
to do the opposite (when the PLATFORM.moduleName happens in an external module we have no control over.
I don't know about that. I think that I need concrete examples to apprehend that better.
Basically you're suggesting that someone splits 3rd party libraries into pieces (at least deployment-wise). This feels like something we shouldn't do and that wasn't supported in previous works. When a library is large and its author wants you to include it in pieces (e.g. a large UI library with many components, or even just jQuery), then the author provides the library in modules that you can join by requesting them individually.
This is possible. But aggressively splitting someone else's library in arbitrary points doesn't seem right to me.
Well... modules designated by
moduleNameare meant to be loaded by Aurelia, whose loader is always async. You can't change that.
You misunderstood. What I mean here is to make the whole chunk load lazily, not the individual module. See bundle-loader to understand what I mean here.
It cannot be a separate plugin concern, because the emitted code of a module will not return the actual module, but a callback which, when called, returns the actual module.
When talking about overriding lazily loaded chunks, we're not talking about jQuery or other non-aurelia packages, but purely about packages/modules which are loaded by the aurelia-loader. The simplest use case is large SPA which is not self-contained, but composes multiple large NPM modules. Modules which contain Aurelia resources, such as templates and models.
But aggressively splitting someone else's library in arbitrary points doesn't seem right to me.
Definitely not talking about that. The simplest use case is chunking a part of an external Aurelia plugin (external module) with, say, 50 custom ValueConverters to lazily load on demand after you navigate to a certain route.
See bundle-loader to understand what I mean here.
From that discussion in #2 I think you don't understand exactly what lazy does. Can you make sure everything is clear on that front and tell me again if it impacts this discussion?
The simplest use case is chunking a part of an external Aurelia plugin (external module) with, say, 50 custom ValueConverters to lazily load on demand after you navigate to a certain route.
If those value converters are only requested by a certain route, and that route is declared with code splitting
PLATFORM.moduleName('route-that-uses-tons-of-converters', 'chunk-xy')
Then my code already works like that. 😃
If you look at my demo 03, the hello-tag custom element used by page-hello was automatically included in the hello.js bundle (that demo has a distinct bundle for each page).
I do understand what bundle-loader?lazy does. The whole loader is extremely simple. In fact, the part responsible for lazy: true is only 5 lines long:
if(query.lazy) {
result = [
"module.exports = function(cb) {\n",
" require.ensure([], function(require) {\n",
" cb(require(", loaderUtils.stringifyRequest(this, "!!" + remainingRequest), "));\n",
" }" + chunkNameParam + ");\n",
"}"];All it does is replaces the module with another one that contains a request to load an external chunk containing the originally requested module and callback() after load. Nothing more, nothing less.
I do think it impacts this discussion, because we still might like to have a centralized config which overwrites this setting. If you do require('bundle-loader?lazy!./some-module') you cannot overwrite this with any plugin, unless you'd do some hackery with dynamically removing loaders. Of course, that's the design of Webpack, because the module passed through a loader does not contain the origin code, but whatever the previous loader returned. But I think operations-wise it's much better to have the option to simply globally turn this off if needed, without having to search&replace all the strings.
With Webpack it's only possible to achieve this for global tests, when you configure rules / loaders in the config file. But here we want to use a loader via the require string, and cannot create that string dynamically (from a configuration file). Does that make it clearer?
If those value converters are only requested by a certain route, and that route is declared with code splitting
PLATFORM.moduleName('route-that-uses-tons-of-converters', 'chunk-xy')Then my code already works like that. 😃
Not in a scenario, where a ValueConverter has its own dependencies which are loaded based on the context. Again, a contrived example could be that a ValueConverter with parameter === 'A' dynamically loads dependency A.js and with parameter === 'B' dynamically loads dependency B.js. Of course, ValueConverters are no good here at all, since they're synchronous by design, but a BindingBehavior isn't synchronous, so perhaps that would be a better choice.
In real life, these could simply be modules with templates and models that are packaged in NPM packages.
A nice example that comes to my mind could be a huge UI library for Aurelia that loads all the UI components dynamically in the index.js file (I think Aurelia-KendoUI does that). For development, we'd want to load all the parts, but for production we might like to list and put unused components through the bundle-loader?lazy, or even blacklist them altogether so their modules return NOOPs.
Not in a scenario, where a ValueConverter has its own dependencies which are loaded based on the context. Again, a contrived example could be that a ValueConverter with parameter === 'A' dynamically loads dependency A.js and with parameter === 'B' dynamically loads dependency B.js.
OK, let's ignore the fact that converters are synchronous.
If I understood correctly:
- the value converter takes a parameter and then loads that module dynamically.
- the page passes 'A' and 'B' as values to that converter.
It would work, simply you need moduleName('A') and moduleName('B') somewhere in that page.
Dependencies will look like:
app -/split/-> page -> tons of value-converter
-> A
-> B
Now that the plugin code has moved to aurelia/webpack-plugin I'm going to close this.
I won't re-open there right now, let's wait for some feedback.
As I recall one demand was being able to create eager chunks from moduleName (split points create lazy chunks).
This can be served in many ways, including DllPlugin, which has long-term caching benefits, or CommonsPlugin. There are many plugins to aggressively merge or split chunks and plugins seem (to me) a better place to configure that rather than in the code itself, which can be quite tricky with multiple dependencies to the same module, etc.
As I said, based on feedback and specific examples we can always reconsider later.