Remove side effects from `quantize()`

[gz65555]

Is your feature request related to a problem? Please describe.
Some models provided by designers need to maintain their original structure, but dedup and prune functions will destroy the structure of the glTF.

Describe the solution you'd like
Add options to disable prune and dedup functions.

Describe alternatives you've considered
Add flags to modified nodes and materials, prune and dedup will only destroy the modified data.

Additional context
The real-world scenario is that designers have a series of models, and we optimize the material size through quantization. These materials of glTF have the same properties in glTF, so they are removed. However, we actually need to adjust these materials at runtime to achieve different effects in different locations.

So I don't want to run dedup function in my glTF process pipeline.

[donmccurdy]

For others' context -

• "prune" removes unused resources
• "dedup" removes duplicate resources

From your description, I understand why de-duplicating materials breaks your use case. Does "prune" also cause a problem, and if so, what?

---

Quantization creates new materials and accessors, and might leave materials or accessors — that were previously unique and in use — as duplicated or unused, unless we do something to clean that up. I don't want to add any user-facing options that would cause new data to be added but not cleaned up. It could double the size of the file, we can't do that.

I think be best solution would be to change quantize() so that instead of calling dedup(), it just cleans up any duplicate resources resulting from its own operation, in some other way. We could do the same for prune() if that's also necessary.

[gz65555]

The prune function also causes a problem:

The designer gave me a well-designed glTF structure without animation, but the animation data are stored in other glTFs (dynamically downloaded according to different scenes). The animation files require the same node tree structure to work. If the prune method is applied,it will result in the node tree of the glTF storing the animations being different from the node tree of the glTF storing the model.

[gz65555]

Sorry, I underestimated the complexity of quantization. I thought quantization only created new nodes.

[donmccurdy]

If the prune method is applied,it will result in the node tree of the glTF storing the animations being different from the node tree of the glTF storing the model.

I think this could be fixed very easily by adding the keepLeaves: true option here:

glTF-Transform/packages/functions/src/quantize.ts

Lines 127 to 134 in 91f4cdf

	await doc.transform(
	prune({
	propertyTypes: [PropertyType.ACCESSOR, PropertyType.SKIN, PropertyType.MATERIAL],
	keepAttributes: true,
	keepIndices: true,
	}),
	dedup({ propertyTypes: [PropertyType.ACCESSOR, PropertyType.MATERIAL, PropertyType.SKIN] }),
	);

I'd be happy to approve a PR for that change if it's useful to you.

The remaining issues with dedup() are harder, though. See #1186. If you need a workaround for that in the meantime, appending some unique data to your materials should avoid the problem:

let id = 1;

for (const material of document.getRoot().listMaterials()) {
  material.setExtras({id: id++});
}

Or maybe we could create a new option like keepUniqueNames so that dedup keeps things with unique names? It can merge things with different names now. Do your materials have unique names already?

[gz65555]

Thank you, it would be very useful to add a keepLeaves option.

As for dedup(), I am also currently avoiding the deletion of materials by adding extra data.

Do your materials have unique names already?

The material names provided by the designer do not follow any special naming convention, which is also difficult to control.

[donmccurdy]

The material names provided by the designer do not follow any special naming convention, which is also difficult to control...

If you wanted to check on this in your assets — I think material names coming from DCC tools like Blender are guaranteed to be unique. That doesn't necessarily mean your three.js materials have unique names (three.js may have to clone some of the original glTF materials) but they'll remain unique in glTF Transform. That'd be a much easier fix than a larger refactor of dedup() at this point.

[gz65555]

Thank you, this is definitely the simplest and most effective solution to this kind of problem. I will adopt this approach.

[donmccurdy]

In progress:

• #1188

[donmccurdy]

I believe this should be mostly solved in #1188, assuming unique materials names, but I'll look a bit more at #1186 as well.