squooshGenerate error: " TypeError: Cannot read properties of undefined (reading 'get')"
pgnd opened this issue · 19 comments
with squooshMinify
module: {
rules: [
{
test: /\.(gif|png|jpe?g|svg|ico)$/,
...
optimization: {
minimize,
minimizer: [
new ImageMinimizerPlugin({
minimizer: {
implementation: ImageMinimizerPlugin.squooshMinify,
options: {
encodeOptions: {
oxipng: {
level: 6,
},
},
},
},
png minification works.
switch, instead, to squooshGenerate, for png->webp generation,
module: {
rules: [
{
test: /\.(gif|png|jpe?g|svg|ico)$/,
...
optimization: {
minimize,
minimizer: [
new ImageMinimizerPlugin({
generator: [
{
implementation: ImageMinimizerPlugin.squooshGenerate,
options: {
encodeOptions: {
webp: {
quality: 90,
},
},
},
},
],
on build, fails
webpack --mode=production --progress --profile
[webpack-cli] HookWebpackError: Cannot read properties of undefined (reading 'get')
at makeWebpackError (/work/test01/.yarn/__virtual__/webpack-virtual-7a36ac1fb4/0/cache/webpack-npm-5.68.0-f34609ad11-ac6efd861a.zip/node_modules/webpack/lib/HookWebpackError.js:48:9)
at /work/test01/.yarn/__virtual__/webpack-virtual-7a36ac1fb4/0/cache/webpack-npm-5.68.0-f34609ad11-ac6efd861a.zip/node_modules/webpack/lib/Compilation.js:3057:12
at eval (eval at create (/work/test01/.yarn/cache/tapable-npm-2.2.1-8cf5ff3039-3b7a1b4d86.zip/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:17:1)
at fn (/work/test01/.yarn/__virtual__/webpack-virtual-7a36ac1fb4/0/cache/webpack-npm-5.68.0-f34609ad11-ac6efd861a.zip/node_modules/webpack/lib/Compilation.js:478:17)
at _next7 (eval at create (/work/test01/.yarn/cache/tapable-npm-2.2.1-8cf5ff3039-3b7a1b4d86.zip/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:14:1)
at eval (eval at create (/work/test01/.yarn/cache/tapable-npm-2.2.1-8cf5ff3039-3b7a1b4d86.zip/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:34:1)
at runMicrotasks (<anonymous>)
at processTicksAndRejections (node:internal/process/task_queues:96:5)
-- inner error --
TypeError: Cannot read properties of undefined (reading 'get')
at WebpackAssetsManifest.handleProcessAssetsAnalyse (/work/test01/.yarn/__virtual__/webpack-assets-manifest-virtual-f2c3bb0a34/0/cache/webpack-assets-manifest-npm-5.1.0-fad26f5290-30b0929f6a.zip/node_modules/webpack-assets-manifest/src/WebpackAssetsManifest.js:467:37)
at fn (/work/test01/.yarn/__virtual__/webpack-virtual-7a36ac1fb4/0/cache/webpack-npm-5.68.0-f34609ad11-ac6efd861a.zip/node_modules/webpack/lib/Compilation.js:476:10)
at _next7 (eval at create (/work/test01/.yarn/cache/tapable-npm-2.2.1-8cf5ff3039-3b7a1b4d86.zip/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:14:1)
at eval (eval at create (/work/test01/.yarn/cache/tapable-npm-2.2.1-8cf5ff3039-3b7a1b4d86.zip/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:34:1)
at runMicrotasks (<anonymous>)
at processTicksAndRejections (node:internal/process/task_queues:96:5)
caused by plugins in Compilation.hooks.processAssets
TypeError: Cannot read properties of undefined (reading 'get')
at WebpackAssetsManifest.handleProcessAssetsAnalyse (/work/test01/.yarn/__virtual__/webpack-assets-manifest-virtual-f2c3bb0a34/0/cache/webpack-assets-manifest-npm-5.1.0-fad26f5290-30b0929f6a.zip/node_modules/webpack-assets-manifest/src/WebpackAssetsManifest.js:467:37)
at fn (/work/test01/.yarn/__virtual__/webpack-virtual-7a36ac1fb4/0/cache/webpack-npm-5.68.0-f34609ad11-ac6efd861a.zip/node_modules/webpack/lib/Compilation.js:476:10)
at _next7 (eval at create (/work/test01/.yarn/cache/tapable-npm-2.2.1-8cf5ff3039-3b7a1b4d86.zip/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:14:1)
at eval (eval at create (/work/test01/.yarn/cache/tapable-npm-2.2.1-8cf5ff3039-3b7a1b4d86.zip/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:34:1)
at runMicrotasks (<anonymous>)
at processTicksAndRejections (node:internal/process/task_queues:96:5)
npx webpack-cli info
System:
OS: Linux 5.16 Fedora Linux 35 (Thirty Five)
CPU: (16) x64 AMD Ryzen 7 3700X 8-Core Processor
Memory: 52.91 GB / 62.70 GB
Binaries:
Node: 17.4.0 - /usr/local/bin/node
Yarn: 3.2.0-rc.13 - /usr/local/bin/yarn
npm: 8.3.1 - /usr/local/bin/npm
Browsers:
Brave Browser: 98.1.35.100
Chrome: 98.0.4758.80
Firefox: 96.0.3
Bug in webpack-assets-manifest
You're right that the issue involves 'manifest'.
But, digging further, I suspect it may be my config/usage, not a bug :-/
I load process Images in Webpack5 using ImageMinimizerPlugin + squoosh
For the simple case, with images imported into .js, and NO manifest,
this now works,
const loadImages = ({ filename, exclude, sizeLimit, minimize }) => ({
module: {
rules: [
{
test: /\.(gif|png|jpe?g|svg|ico)$/,
exclude,
type: 'asset/resource',
generator: { filename, },
parser: { dataUrlCondition: { maxSize: sizeLimit }, },
},
],
},
optimization: {
minimize,
minimizer: [
new ImageMinimizerPlugin({
generator: [
{
// preset: "webp",
type: "asset", <=========== adding this cures the 'get' error from above
implementation: ImageMinimizerPlugin.squooshGenerate,
options: { encodeOptions: { webp: { quality: 90, }, }, },
},
],
/*
minimizer: {
implementation: ImageMinimizerPlugin.squooshMinify,
options: {
encodeOptions: {
oxipng: { level: 6, },
mozjpeg: { quality: 100, },
webp: { lossless: 1, },
avif: { cqLevel: 0, },
},
},
},
*/
}),
],
},
});
When i add @webdeveric 's webpack-assets-manifest (https://github.com/webdeveric/webpack-assets-manifest) usage,
...
plugins: [
new WebpackAssetsManifest({
enabled: true,
...
and switch to the generator, there's problems.
WITH asset creation in place, and the ImageMinimizerPlugin generator DISABLED & minimizer ENABLED, existing .png resource assets are correctly processed, and added to the asset manifest.
e.g., after a webpack production build,
ls -al public/build/img/TEST*
-rw-rw-r--+ 1 wwwrun www 9.7K Feb 6 19:50 public/build/img/TEST.e21cfc6f2d5e08ffcc5d.png
grep TEST public/build/assets-manifest.json -A2
"TEST.png": {
"src": "/build/img/TEST.e21cfc6f2d5e08ffcc5d.png",
},
and, in site, the asset is referenced as, simply, the unhashed manifest key,
TEST.png
if I simply ENABLE the generator, DISABLE the minimizer, and production build, there are, again, no ERRORS in the build
result is,
ls -al public/build/img/TEST*
-rw-rw-r--+ 1 wwwrun www 8.0K Feb 7 07:57 public/build/img/TEST.57c1158370a82c33f393.webp
file public/build/img/TEST*
public/build/img/TEST.57c1158370a82c33f393.webp: RIFF (little-endian) data, Web/P image
grep TEST public/build/assets-manifest.json -A2
"TEST.57c1158370a82c33f393.webp": {
"src": "/build/img/TEST.57c1158370a82c33f393.webp",
},
NOTE that the manifest's image key now contains the hash!
So, the image is NOT referenceable in site by the expected
TEST.webp
but only by the existing
TEST.57c1158370a82c33f393.webp
When adding the 'extra' step of ImageMinimizerPlugin webp squoosh-generation, the hash gets "added" to the manifest key.
Is that corrected in ImageMinimizerPlugin config? Squoosh? or the manifest plugin?
Because WebpackAssetsManifest doesn't support renamed assets, sorry we can't fix it here, if you provide github repo I will show there is problems happens and how to fix it
Because
WebpackAssetsManifestdoesn't support renamed assets, sorry we can't fix it here, if you provide github repo I will show there is problems happens and how to fix it
fyi, i've provided reproducer over @
I will show there is problems happens and how to fix it
could you suggest that fix? either here, or at the
webdeveric/webpack-assets-manifest#219 (comment)
?
I think I known why it happens, can you provide reproducible test repo, so I will check it
Oh, seems you have https://github.com/pgnd/test01.git, give me time
Hm, I found how to fix it, but why you expected TEST.webp webdeveric/webpack-assets-manifest#219 (comment), I think it should be TEST.png, because original file is TEST.png, so after converting and adding hashes it will be /build/img/TEST.d3453ac871872dfc247c.webp, but it is interesting edge case, you can have TEST.png and TEST.webp in one application, so you can have two same keys.
We need to found better approach for this case here and I will fix it
I 'expected' TEST.webp, because I'm using the squoosh generator, which does a conversion -- from .png (source) -> .webp (packaged).
Using it as,
new ImageMinimizerPlugin({
generator: [
{
type: "asset",
implementation: ImageMinimizerPlugin.squooshGenerate,
options: {
encodeOptions: {
webp: {
quality: 90,
},
},
},
},
],
Using it, that's in fact what created -- a .webp. As intended.
In this case, the [hash] is incorrectly added to the manifest key.
OTOH, if I switch BACK to only using the minimizer,
new ImageMinimizerPlugin({
minimizer: {
implementation: ImageMinimizerPlugin.squooshMinify,
options: {
encodeOptions: {
oxipng: {
level: 6,
},
webp: {
lossless: 1,
},
},
},
},
],
then the webpack build does what is should, and does it correctly.
Namely it sources an image file (.png or .webp), packs/minimizes it in same, unconverted format (i.e., png -> png, & webp -> webp -- NO cross conversion), hashes the file name, and correctly adds the hashed path+filename to the manifest as value, for the UN-hashed key.
In this case, the [hash] is incorrectly added to the manifest key.
The problem is not in hash, you see it due another root problem, when you enable image generation, i.e. png -> webp we don't pass sourceFilename (i.e. original filename), but original filename is TEST.png https://github.com/webdeveric/webpack-assets-manifest/blob/master/src/WebpackAssetsManifest.js#L573, yes we can rename it to webp and it is not a problem, but I think it can invalid, because sourceFilename is TEST.png and it can be confusing, unfortunately WebpackAssetsManifest doesn't support other info properties.
That is why I paid attention to it
Hm.
It's not simply an issue of re-naming, is it?
The source file IS a .png,
file ./resources_loc/assets/img/TEST.png
PNG image data, 1280 x 853, 8-bit/color RGB, non-interlaced
and the generated file IS a .webp
file ./public/build/img/TEST.d3453ac871872dfc247c.webp:
RIFF (little-endian) data, Web/P image, VP8 encoding, 1280x853, Scaling: [none]x[none], YUV color, decoders should clamp
and that's the ONLY file generated -- i.e. TEST.[hash].webp
so, iiuc, the manifest correctly must reference "TEST.webp" as the key, for the path to the generated file value.
in layout source, then, the generated asset is correctly referenced as the key, "TEST.webp"
perhaps I'm misunderstanding your point ... ?
@alexander-akait What other info properties should I support? If there are any other commonly used properties besides sourceFilename, then I'd be willing to add support for those.
@pgnd Since the .png is the original source for the generated .webp, I think the key should be the .png file, but I'm open to feedback based on how other devs want it to behave.
For all assets in a build, (regardless if generated or not), I'll try to use info.sourceFilename if it exists and fallback to the generated filename for the manifest key.
My intended plan, at least, was to structure my project with sources of any image type (png, webp, jpeg, gif, etc), and use squoosh generator to convert all source types to webp generated, hashed (cache-bustable) assets.
Then, in my layout, to reference all image assets as the UNHASHED .webp ( `< img src="test.webp" ...> ), which, when retrieved by key:value reference from the manifest, would correctly render the hashed file.
That's exactly how source -> generated mapping occurs in the project manifest for the NON-conversion case.
I'm just advocating that that is ALSO how it should work for the case where I do conversions.
The conversion is done @ build time.
What's deployed (e.g., promoted from staging -> production site) is ONLY the set of built assets -- in this case, the webp images.
Manifest maps intransient key to transient (hashed) asset name. That's all.
@webdeveric Yep, that is why I am thinking it should be TEST.png, but developer can generates more assets and keep some of them as is, i.e. we can have image.[hash].png/image.[hash].web/image.[hash].avif, so for both we will have image.png key and it is a collision for JSON (because we just override previously key).
From DX experience make sense to have image.[original_ext] because it will not create collisions and easy to use, on the other hand when I ask image.png I should have ability to choose.
Ideally I see it (pseudo code):
const webp = manifest["image.png"]["generated"]["webp"]["filename"];
And even more
const original = manifest["image.png"]["filename"];
const webp = manifest["image.png"]["generated"]["webp"]["filename"];
const avif = manifest["image.png"]["generated"]["avif"]["filename"];
let element =
`<picture>
<source srcset="${avif}" type="image/avif">
<source srcset="${webp}" type="image/webp">
<img src="${original}" alt="logo">
</picture>`
But asset manifest outputs 1:1, i.e. { "image.[hash].png": "image.png" }
@pgnd Again, the problem is not in hash, we can drop hash and do what your expected, but it can creates another problems, what is why we need discussion here, for example you can have image.webp (using copy-webpack-plugin) and image.png and generate webp from image.png, and you will have collision in your assets manifest.
@webdeveric In theory we can do it:
{
"file.png": "file.[hash].png",
"mapping": {
"file.png": {
"webp" => "file.[hash].png",
"avif" => "file.[hash].avif"
}
}
}
Ideally format of assets manifest should be another, i.e.
{
"image.png": {
"filename": "image.[hash].png"
}
}
So we will have ability to add any numbers of fields - generated, size and even more, so developer can construct any complex logic around, for example I can decide to change output based on size of asset when I do SSR.
But I understand it will be big breaking change. So I want to hear your ideas.
@pgnd Please read the problem carefully, we try to protect assets manifest from collisions, you can always change logic of getWebpackAsset, it is not hard, we can't just use TEST.webp as you want to see, because it is unsafe and wrong
@alexander-akait I'll think on how to keep and manage the associations of source file to generated files. I'll try to work on it after work today.
@webdeveric Feel free to ping me, we can do it together