Configuring image quantization quality to optimize performance

Image quantization may take a significant amount of time (especially for high resolution images).

The #129 improves things a bit, but it just improving the UI responsiveness by moving the extraction of the dominant colors of the image form the main thread to web worker. It does not speed up the process per se.

BTW, it seems using WASM (see #99) or GLSL or WGSL (see #128) could improve the performance dramatically.

Another way to improve performance of image quantization is to lower the resolution of the input image.

I see this as the second quality argument of the sourceColorFromImage function (as is done in the color-thief — https://lokeshdhakar.com/projects/color-thief/#api)

material-color-utilities/typescript/utils/image_utils.ts

Lines 29 to 79 in f5d03da

    
           export async function sourceColorFromImage(image: HTMLImageElement) { 
        
             // Convert Image data to Pixel Array 
        
             const imageBytes = await new Promise<Uint8ClampedArray>((resolve, reject) => { 
        
               const canvas = document.createElement('canvas'); 
        
               const context = canvas.getContext('2d'); 
        
               if (!context) { 
        
                 reject(new Error('Could not get canvas context')); 
        
                 return; 
        
               } 
        
               const callback = () => { 
        
                 canvas.width = image.width; 
        
                 canvas.height = image.height; 
        
                 context.drawImage(image, 0, 0); 
        
                 let rect = [0, 0, image.width, image.height]; 
        
                 const area = image.dataset['area']; 
        
                 if (area && /^\d+(\s*,\s*\d+){3}$/.test(area)) { 
        
                   rect = area.split(/\s*,\s*/).map(s => { 
        
                     // tslint:disable-next-line:ban 
        
                     return parseInt(s, 10); 
        
                   }); 
        
                 } 
        
                 const [sx, sy, sw, sh] = rect; 
        
                 resolve(context.getImageData(sx, sy, sw, sh).data); 
        
               }; 
        
               if (image.complete) { 
        
                 callback(); 
        
               } else { 
        
                 image.onload = callback; 
        
               } 
        
             }); 
        
             // Convert Image data to Pixel Array 
        
             const pixels: number[] = []; 
        
             for (let i = 0; i < imageBytes.length; i += 4) { 
        
               const r = imageBytes[i]; 
        
               const g = imageBytes[i + 1]; 
        
               const b = imageBytes[i + 2]; 
        
               const a = imageBytes[i + 3]; 
        
               if (a < 255) { 
        
                 continue; 
        
               } 
        
               const argb = argbFromRgb(r, g, b); 
        
               pixels.push(argb); 
        
             } 
        
             // Convert Pixels to Material Colors 
        
             const result = QuantizerCelebi.quantize(pixels, 128); 
        
             const ranked = Score.score(result); 
        
             const top = ranked[0]; 
        
             return top; 
        
           }

This can be implemented in two ways:

By reducing the size of the canvas. BTW, The maximum size should be limited in any case.
By skipping every n-th pixel. We rarely need to sample every single pixel in the image to get good results.

This should also fix the #130 bug.

I could prepare a pull request if you are interested, but though only for TypeScript implementation. 😉

Obviously, implementations in other languages don't use browser canvas like the TypeScript implementation does. So I think we should use the second approach with pixel skipping. Right?

But first it would be nice to fix the #132 bug to prove that lowering the resolution of the input image doesn't affect the result.

@rodydavis @guidezpl PTAL.

@pennzht @marshallworks PTAL.

	export async function sourceColorFromImage(image: HTMLImageElement) {
	// Convert Image data to Pixel Array
	const imageBytes = await new Promise<Uint8ClampedArray>((resolve, reject) => {
	const canvas = document.createElement('canvas');
	const context = canvas.getContext('2d');
	if (!context) {
	reject(new Error('Could not get canvas context'));
	return;
	}
	const callback = () => {
	canvas.width = image.width;
	canvas.height = image.height;
	context.drawImage(image, 0, 0);
	let rect = [0, 0, image.width, image.height];
	const area = image.dataset['area'];
	if (area && /^\d+(\s,\s\d+){3}$/.test(area)) {
	rect = area.split(/\s,\s/).map(s => {
	// tslint:disable-next-line:ban
	return parseInt(s, 10);
	});
	}
	const [sx, sy, sw, sh] = rect;
	resolve(context.getImageData(sx, sy, sw, sh).data);
	};
	if (image.complete) {
	callback();
	} else {
	image.onload = callback;
	}
	});

	// Convert Image data to Pixel Array
	const pixels: number[] = [];
	for (let i = 0; i < imageBytes.length; i += 4) {
	const r = imageBytes[i];
	const g = imageBytes[i + 1];
	const b = imageBytes[i + 2];
	const a = imageBytes[i + 3];
	if (a < 255) {
	continue;
	}
	const argb = argbFromRgb(r, g, b);
	pixels.push(argb);
	}

	// Convert Pixels to Material Colors
	const result = QuantizerCelebi.quantize(pixels, 128);
	const ranked = Score.score(result);
	const top = ranked[0];
	return top;
	}