webpack-dedupe-plugin in watch mode with mini-css-extract-plugin

[7rulnik]

Hello from Aviasales :)

We are trying to use webpack-dedupe-plugin for local development, especially with yarn link. As I can see right now it works only in production mode because normalModuleFactory called only once https://github.com/Tinkoff/tramvai/blob/main/packages/libs/webpack-dedupe-plugin/src/DedupePlugin.ts#L201-L204

We tried to adopt it for watch mode with mini-css-extract-plugin. It doesn't use a child compiler anymore, but we faced a bug: currently within the cache key could be stored different modules with different loaders.

[
    "@csstools/normalize.css@12.0.0:./normalize.css",
    [
        "/Users/v7rulnik/projects/aviasales/selene/node_modules/mini-css-extract-plugin/dist/loader.js!/Users/v7rulnik/projects/aviasales/selene/node_modules/css-loader/dist/cjs.js??ruleSet[1].rules[5].use[1]!/Users/v7rulnik/projects/aviasales/selene/node_modules/@csstools/normalize.css/normalize.css",
        "/Users/v7rulnik/projects/aviasales/selene/node_modules/css-loader/dist/cjs.js??ruleSet[1].rules[5].use[1]!/Users/v7rulnik/projects/aviasales/selene/node_modules/@csstools/normalize.css/normalize.css"
    ]
]

So we added loaders to cache keys:

diff --git a/lib/DedupePlugin.js b/lib/DedupePlugin.js
index 46fefe8e75b7435c0165d3cdbe622253aebdb122..3cd56486a83f3d3e3ea930b89462e7d8c5a12114 100644
--- a/lib/DedupePlugin.js
+++ b/lib/DedupePlugin.js
@@ -128,14 +128,7 @@ class DedupePlugin {
         }
     }
     apply(compiler) {
-        let called = false;
-        // этот хук будет вызываться при каждом создании компилятора, но это создает дубли модулей,
-        // с которыми пока непонятно что делать, поэтому просто игнорируем childCompiler
         compiler.hooks.normalModuleFactory.tap(PLUGIN_NAME, (nmf) => {
-            if (called) {
-                return;
-            }
-            called = true;
             nmf.hooks.module.tap(PLUGIN_NAME, (module, result) => {
                 // игнорируем все что вне node_modules, т.к. вряд-ли это нужно дедуплицировать или если модуль игнорируем
                 if (!module.resource.includes('node_modules') || this.isIgnoredModule(result)) {
@@ -225,8 +218,10 @@ function createCacheKeyFromNMFResult(nmfResult, useSemver = false) {
     // если semver то используем только мажорную версию пакета
     // если нет, то оставляем полную версию
     const versionForCache = useSemver ? buildCacheVersionFromNMFResult(nmfResult) : version;
+    const loaders = nmfResult.loaders.map(item => item.loader).join('|');
     // ключ = имя + версия + путь к исходному файлу относительно корня либы
-    return `${name}@${versionForCache}:${nmfResult.resourceResolveData.relativePath}`;
+    return `${name}@${versionForCache}:${nmfResult.resourceResolveData.relativePath}:${loaders}`;
 }
 /**
  * Получаем основную версию для зависимости по правилу semver `^version`

I understand that you have your own stack and needs so basically I created this issue to give some feedback. But maybe you see how we can support our (and I believe many people too) use case?

[meskill]

Hello, thank you for interest in our plugin.

I'm trying to understand of use case for this plugin in dev mode. As you mentioned yarn link I guess you want to bypass some issues coming when usual and linked dependencies get resolved with different version of some shared library e.g. React. Am I right?

[7rulnik]

@meskill yeah, react is a great example of it.

[meskill]

Have you considered use other tools like yalc to bypass the yarn link issues?

As for plugin, our main focus was only the production build size reduction and I haven't thought of using it in development. I need to investigate a little more and check that your suggestion to use loaders in cache key is safe. Have you tested updated version in production builds?

[7rulnik]

@meskill yeah, I just tested it in prod mode. No difference between original and patched versions (I committed the whole dist directory to check it).

Firstly I was thinking about pnpm but it still has the same problems with link.

And also I checked yalc, but I think nothing should touch node_modules except the package manager itself. But the main problem, that developers still need to push changes manually (at least as I understand it). As compromise we can solve it via yarh pack and install local tar which feels like same thing that yalc does.

[meskill]

Ok, we'll think about integrating your suggestion as it might be helpful in some use-cases. thanks for investigating and explanations

[SuperOleg39]

Hi!

@7rulnik, suggested fix was integrated by @meskill and released with 2.93.0 tramvai.

Thanks for your contribution!

[7rulnik]

Thanks, guys! <3