webpack5: getPathKey returns key not in bundlesContent; leads to sha1(undefined) in karma
himdel opened this issue · 17 comments
- Operating System: linux
- Node Version: 14
- NPM Version: yarn 1.22
- webpack Version: 5.3.0 (webpack-cli 4.1.0)
- karma-webpack Version: #next, 5.0.0-alpha.3.0
Expected Behavior
using './client/app.js': ['webpack'] in karma.conf preprocessors, processing that file should work.
(With an otherwise working webpack config.)
Actual Behavior
27 10 2020 21:49:28.041:ERROR [karma-server]: UnhandledRejection: The "data" argument must be of type string or an instance of Buffer, TypedArray, or DataView. Received undefined
27 10 2020 21:49:28.042:ERROR [karma-server]: TypeError [ERR_INVALID_ARG_TYPE]: The "data" argument must be of type string or an instance of Buffer, TypedArray, or DataView. Received undefined
at Hash.update (internal/crypto/hash.js:84:11)
at Object.sha1 (/home/himdel/manageiq-ui-service/node_modules/karma/lib/utils/crypto-utils.js:9:8)
at runProcessors (/home/himdel/manageiq-ui-service/node_modules/karma/lib/preprocessor.js:82:26)
at async FileList.preprocess [as _preprocess] (/home/himdel/manageiq-ui-service/node_modules/karma/lib/preprocessor.js:146:5)
at async Promise.all (index 0)
at async /home/himdel/manageiq-ui-service/node_modules/karma/lib/file-list.js:90:11
at async Promise.all (index 9) {
code: 'ERR_INVALID_ARG_TYPE'
in karma/lib/preprocessor.js, the failing file is '/home/himdel/manageiq-ui-service/client/app.js' (the only file using the webpack preprocessor),
initial content before the webpack preprocessor is the file content, after the webpack preprocessor, it's undefined.
In karma-webpack/lib/karma-webpack.js, after await controller.bundle() in processFile:
bundleContent is undefined,
getPathKey(file.path, true) returns app.1564016497.js,
while the only keys matching /app/ in controller.bundlesContent are:
'sass/_app_urls.sass',
'sass/_app_colors.sass',
'js/app.1564016497-3d658aca95fc6995589c.js',
'js/app-208bf5b45fa818efa9cf.js'
So looks like webpack5 is adding a prefix and a hash suffix, which getPathKey doesn't handle.
Do I need to override some part of webpack config so that it doesn't do that for tests, or is this an actual bug?
(The fix in #451 doesn't help, but that's not surprising as this is pure js, not typescript.)
What did help in my case was:
- const bundleContent =
- controller.bundlesContent[getPathKey(file.path, true)];
+ let key = `js/${getPathKey(file.path)}-`;
+ key = Object.keys(controller.bundlesContent).find((name) => name.startsWith(key));
+ const bundleContent = controller.bundlesContent[key];So looks like webpack5 is adding a prefix and a hash suffix, which getPathKey doesn't handle.
Ok, it is coming from the webpack config, via output.filename.
So... I can fix this on my side, by explicitly forcing config.output.filename = '[name].js' in webpack testing config.
Feel free to close this .. but maybe you want to override that / warn about it / handle that case?
It seems that this error happens in a few cases when the webpack configuration has an invalid field, I have a similar issue over here. #458. I think we can treat this in the same way, I just want to verify that these are required settings and that we won't break anyones environment by setting these automatically and warning about invalid configuration.
To limit confusion, I am combining all threads for webpack 5 / karma6 support over here ==> #475
v5 is now available
I'm using webpack 5.28.0 and karma-webpack 5.0.0 and I'm still getting the TypeError [ERR_INVALID_ARG_TYPE]: The "data" argument must be of type string or an instance of Buffer, TypedArray, or DataView. Received undefined error when using "declaration": true in the compiler options. Is there some specific karma config I need to have to prevent that error?
I'm using webpack 5.38.1 and karma-webpack 5.0.0-alpha.6 and still getting
09 06 2021 14:11:35.686:ERROR [karma-server]: Error during file loading or preprocessing
TypeError [ERR_INVALID_ARG_TYPE]: The "data" argument must be of type string or an instance of Buffer, TypedArray, or DataView. Received undefined
at Hash.update (internal/crypto/hash.js:84:11)
at Object.sha1 (/Users/chenziyun/workspace/netease/im/sdk/node_modules/karma/lib/utils/crypto-utils.js:9:8)
My karma conf
const path = require('path')
module.exports = function(config) {
config.set({
// base path that will be used to resolve all patterns (eg. files, exclude)
basePath: '',
// mime: {
// 'text/x-typescript': ['ts', 'tsx']
// },
frameworks: ['mocha', 'chai', 'webpack'],
// list of files / patterns to load in the browser
files: [
// { pattern: '', included: false },
// 'dist/NIM_BROWSER_SDK.js',
// 'test/**/*.spec.js'
'test/index.ts'
],
// list of files / patterns to exclude
exclude: [
// 'dist/NIM_BROWSER_SDK.js'
],
// preprocess matching files before serving them to the browser
// available preprocessors: https://www.npmjs.com/search?q=keywords:karma-preprocessor
// preprocessors: {
// 'test/**/*.spec.js': ['webpack']
// },
preprocessors: {
'test/index.ts': ['webpack']
},
// test results reporter to use
// possible values: 'dots', 'progress'
// available reporters: https://www.npmjs.com/search?q=keywords:karma-reporter
// reporters: ['nyan', 'coverage-istanbul'],
reporters: ['progress'],
// web server port
port: 9876,
// enable / disable colors in the output (reporters and logs)
colors: true,
// level of logging
// possible values: config.LOG_DISABLE || config.LOG_ERROR || config.LOG_WARN || config.LOG_INFO || config.LOG_DEBUG
logLevel: config.LOG_INFO,
autoWatch: true,
browsers: ['Chrome'],
singleRun: !!process.env.CI,
concurrency: Infinity,
webpackMiddleware: {
noInfo: true,
stats: 'errors-only'
},
webpack: {
mode: 'development',
entry: './src/entry/browser.ts',
output: {
filename: '[name].js'
},
devtool: 'inline-source-map',
module: {
rules: [
{
test: /\.tsx?$/,
use: {
loader: 'ts-loader',
options: {
configFile: 'test/tsconfig.json'
}
},
exclude: [path.join(__dirname, 'node_modules')]
},
]
},
resolve: {
extensions: ['.tsx', '.ts', '.js', '.json']
}
},
})
}My test file in test/index.ts
const tests = require.context('./', true, /\.spec\.tsx?$/)
tests.keys().forEach(tests)
const sources = require.context('../src/', true, /\.tsx?$/)
sources.keys().forEach(sources)
So looks like webpack5 is adding a prefix and a hash suffix, which getPathKey doesn't handle.
Ok, it is coming from the webpack config, via
output.filename.So... I can fix this on my side, by explicitly forcing
config.output.filename = '[name].js'in webpack testing config.Feel free to close this .. but maybe you want to override that / warn about it / handle that case?
I set config.output.filename = '[name].js' in webpack testing config，no use。 I print getPathKey and controller.bundlesContent in karma-webpack/lib/karma-webpack.js
getPathKey~~~~~~~~~~ index.3934682155.ts
processFile～～～～～～～～ [ 'commons.js', 'runtime.js' ]
It's weird. My karma config is in the comments above
So looks like webpack5 is adding a prefix and a hash suffix, which getPathKey doesn't handle.
Ok, it is coming from the webpack config, via
output.filename.
So... I can fix this on my side, by explicitly forcingconfig.output.filename = '[name].js'in webpack testing config.
Feel free to close this .. but maybe you want to override that / warn about it / handle that case?I set
config.output.filename = '[name].js'in webpack testing config，no use。 I printgetPathKeyandcontroller.bundlesContentinkarma-webpack/lib/karma-webpack.jsgetPathKey~~~~~~~~~~ index.3934682155.ts processFile～～～～～～～～ [ 'commons.js', 'runtime.js' ]It's weird. My karma config is in the comments above
Did you update to karma-webpack v5.0.0 ?
I had the same issue with karma-webpack v5.0.0, but only on macOS. No issue whatsoever on Linux & Windows machines.
Changing this line:
to return bundleContent; made this error message go away (I still have another issue to fix). I can do a PR with this change if you want, but I'm not sure I understand fully what is going on here. Seems related to https://github.com/karma-runner/karma/blob/2b71a3c6a01757e0b5c1bf8d241b588656b62127/lib/preprocessor.js#L47-L55
The changes bewteen master and v5.0.0 also fix this issue, could you do a v5.0.1 release with them?
Yes, I think these changes are the ones: ef7edb6#diff-282d7a470af0c4d49f499674bcd6d416cdc3ad49918bb564a2ae5e3e8dad81ccR86-R96
Yes, I think these changes are the ones: ef7edb6#diff-282d7a470af0c4d49f499674bcd6d416cdc3ad49918bb564a2ae5e3e8dad81ccR86-R96
Strange, that doesn't actually change anything. It just replaces async with promises.
I think the error is related to async/promise and the code in https://github.com/karma-runner/karma/blob/2b71a3c6a01757e0b5c1bf8d241b588656b62127/lib/preprocessor.js#L47-L55, especially the branch doing return donePromise, which ends up returning undefined, and triggering the error The "data" argument must be of type string or an instance of Buffer, TypedArray, or DataView. Received undefined.
From v5.0.0/e2423b47ec59059bc1808c670affa64d0f5aa8b7, changing
to
return bundleContent; also fixes the error.
My understanding is that async function are implicitly wrapped in promise, and since the function on v5.0.0 did not return (i.e. it returned undefined), that's what karma was seeing. On master, the code is explicitly using a promise (so no implicit wrapping of the function inside another promise occurs), and so calling done actually transfer the control flow back to the outer promise.
Update
Solved with removing some entries from preprocessors from karma config.
preprocessors: {
'src/**/*Spec.js': ['webpack'],
- 'test/**/*.js': ['webpack']
},Original comment
Similar issue here. Using:
npm ls webpack karma karma-webpack
├── karma@6.3.4
├── karma-webpack@5.0.0
└── webpack@5.34.0
getting:
START:
Webpack bundling...
238 assets
webpack 5.34.0 compiled successfully in 27074 ms
18 09 2021 18:13:10.990:ERROR [karma-server]: Error during file loading or preprocessing
TypeError [ERR_INVALID_ARG_TYPE]: The "data" argument must be of type string or an instance of Buffer, TypedArray, or
DataView. Received undefined