Build failed due to missing `readable-stream` files

[thinkh]

With PR Caleydo/tdp_publicdb#118 the package madge was removed from the dev dependencies.

Since then the Ordino product build fails with the following CircleCI build errors:

ERROR in ./node_modules/stream-browserify/index.js
Module not found: Error: Can't resolve 'readable-stream/duplex.js' in '/home/circleci/phovea/tmp0_ordin/node_modules/stream-browserify'
 @ ./node_modules/stream-browserify/index.js 30:16-52
 @ ./node_modules/cipher-base/index.js
 @ ./node_modules/create-hmac/browser.js
 @ ./node_modules/crypto-browserify/index.js
 @ ./node_modules/crypto-js/core.js
 @ ./node_modules/crypto-js/md5.js
 @ ./tdp_matomo/dist/app/Matomo.js
 @ ./tdp_matomo/dist/phovea.js
 @ ./tdp_matomo/phovea_registry.js
 @ ./phovea_registry.js
 @ multi ./phovea_registry.js ./ordino_public/dist/initialize.js

ERROR in ./node_modules/stream-browserify/index.js
Module not found: Error: Can't resolve 'readable-stream/passthrough.js' in '/home/circleci/phovea/tmp0_ordin/node_modules/stream-browserify'
 @ ./node_modules/stream-browserify/index.js 32:21-62
 @ ./node_modules/cipher-base/index.js
 @ ./node_modules/create-hmac/browser.js
 @ ./node_modules/crypto-browserify/index.js
 @ ./node_modules/crypto-js/core.js
 @ ./node_modules/crypto-js/md5.js
 @ ./tdp_matomo/dist/app/Matomo.js
 @ ./tdp_matomo/dist/phovea.js
 @ ./tdp_matomo/phovea_registry.js
 @ ./phovea_registry.js
 @ multi ./phovea_registry.js ./ordino_public/dist/initialize.js

ERROR in ./node_modules/stream-browserify/index.js
Module not found: Error: Can't resolve 'readable-stream/transform.js' in '/home/circleci/phovea/tmp0_ordin/node_modules/stream-browserify'
 @ ./node_modules/stream-browserify/index.js 31:19-58
 @ ./node_modules/cipher-base/index.js
 @ ./node_modules/create-hmac/browser.js
 @ ./node_modules/crypto-browserify/index.js
 @ ./node_modules/crypto-js/core.js
 @ ./node_modules/crypto-js/md5.js
 @ ./tdp_matomo/dist/app/Matomo.js
 @ ./tdp_matomo/dist/phovea.js
 @ ./tdp_matomo/phovea_registry.js
 @ ./phovea_registry.js
 @ multi ./phovea_registry.js ./ordino_public/dist/initialize.js

ERROR in ./node_modules/stream-browserify/index.js
Module not found: Error: Can't resolve 'readable-stream/writable.js' in '/home/circleci/phovea/tmp0_ordin/node_modules/stream-browserify'
 @ ./node_modules/stream-browserify/index.js 29:18-56
 @ ./node_modules/cipher-base/index.js
 @ ./node_modules/create-hmac/browser.js
 @ ./node_modules/crypto-browserify/index.js
 @ ./node_modules/crypto-js/core.js
 @ ./node_modules/crypto-js/md5.js
 @ ./tdp_matomo/dist/app/Matomo.js
 @ ./tdp_matomo/dist/phovea.js
 @ ./tdp_matomo/phovea_registry.js
 @ ./phovea_registry.js
 @ multi ./phovea_registry.js ./ordino_public/dist/initialize.js

Investigating the problem shows that removing the madge dependency switched the version of readable-stream from version 2.x to version 3.x in the node_modules root. Instead the version 2.x dependencies of multiple packages got nested and resulting in broken imports.

Reading about the functionality of npm's dependency resolution, I found this quote:

The key is to remember that install order matters.

With this in mind our build was previously succeeding because madge uses readable stream in version 2.x (and set this version as default). The package-lock.json (with just installing madge as dependency) confirms this theory.

Taking a closer look at the workspace package-lock.json shows that the version 3.x is introduced by webpack:

webpack@4.43.0
node-libs-browser@2.2.1
crypto-browserify@3.12.0
browserify-sign@4.2.1
readable-stream@3.6.0 deduped

The only option to fix this build error is to explicitly add readable-stream in version 2.x to one of our plugins. Since the error points to the tdp_matomo repository, I'd suggest to add it there. In my local tests the webpack build succeeds, once readable-stream@^2.0.0 is added as dev dependency to the workspace.

[thinkh]

The CircleCI build 187 with the the merged PR datavisyn/tdp_matomo#34 is sucessfull again.