TSL: Missing a uvNode to rewrite the UV's
Samsy opened this issue · 8 comments
Description
Were looking for a uvNode to rewrite the UV's for custom purpose but could not find a way to override the uv's ideally at different stage : in both vertex and fragment stage separately
I was some problem with the approach bellow?
const customUV = uv();
const sample = texture( map, customUV );Could you provide more details?
Mostly, there's a need to custom the UV's in the vertex stage to avoid computing them-per pixel, but more importantly it re-join the conversation of extending an existing a node of a built-in material without having to re-writing it all
For a simple example, using a basicmaterial, If I need to spritesheet the UV's, I'd just need to re-write the UV's in the vertex shader, and would not have the need of the snippet you provided because the rest would follow along without even knowing if the material got a texture assigned or not
const customUV = uv();
const sample = texture( map, customUV );
But also for a more complex example, a standard material, as a user I don't know what are the implications and what is inside a default built-in standardmaterial color node, for this I need knowledge of the actual default node and re-write it entirely and incorporate const customUV = uv(); inside the logic along with the rest of the default built-in color node
All of this mostly because as a user, I'm looking to write a node that could work in any case means without breaking the built-in behavior, without having knowledge of the rests of the node, and what ever the next nodes would be
Using the legacy system, I would just replace the chunk that was dealing with uvs in the vertex, no matter the rest of the shader, if it is using a texture, or applying some calcs with UV's as args, just extending the built-in behavior :
vertexShader.replace(
"vUv = uv;",
"vUv = customUV(uv);
) etc..
And the same applies in a previous conversation we had about rotating normals, here a snippet, the only way to rotate the normals in the vertex stage, is to rotate them inside a positionNode, but this needs knowledge of the actual built-in positionNode behavior to re-write it and correctly output the position on top of the normal behavior :
material.positionNode = Fn(() => {
const pos = attribute('position', 'vec3').toVar();
const rotateMatrix = rotateY( time );
normalLocal.assign( rotateMatrix.mul( normalLocal ) )
return pos
})();
This rejoin the conversation about Extending a built-in material :
In finé, we are missing a way to update actual raw attributes in vertex stage :
Custom the raw UV's versus custom the sampling coordinate for a specific texture
Custom the raw normal versus custom a fully compatible normalNode in a fragment shader ( or hacking into a positionNode to set the normalLocal value )
Using storageBuffers instead of attributes is just how I imagine it. You can change these as you wish with comute shaders and use them in vertex shaders as attributes, which is what I do. But that means pure WebGPU with three.webgpu.js
You can compute any node/math in vertex stage using varying(), e.g:
const customUV = rotate( uv(), time ).mul( 2 ).varying(); // rotate(...).mul(2) will be computed in vertex stage
const sample = texture( map, customUV );The legacy uv is the same as TSL's uv(). Instead, share the same UV with the other textures you will use.
const sampleA = texture( mapA, customUV );
const sampleB = texture( mapB, customUV );The node system in general, not only in TSL, tends to avoid global replacements because it prioritizes API stability. You will have to create some Node or another to achieve the desired effect, this is part of the new approach.
Custom the raw normal versus custom a fully compatible normalNode in a fragment shader ( or hacking into a positionNode to set the normalLocal value )
normalLocal and .normalNode have different purposes, while one takes care of the transformation related to the geometry, the other takes care of the normals related to the surface generated by the vertex stage.
Using the legacy system, I would just replace the chunk that was dealing with uvs in the vertex, no matter the rest of the shader, if it is using a texture, or applying some calcs with UV's as args, just extending the built-in behavior :
vertexShader.replace(
"vUv = uv;",
"vUv = customUV(uv);
) etc..
The simple UV replacement as mentioned at a global level will only work in older versions of Three.js, after the updates of different UV index and independent Matrices per texture, every texture has its own UV like vMapUv, vAlphaMapUv, etc. which also does not fit into hack global replacement instead you can modify directly on the Texture for spritesheet like:
//const map = new Texture();
map.channel = ...
map.offset = ...
map.repeat = ...
...Or you can create a function to do this with TSL, to just replace the current map logic with another one for example. What would be correct if you want to have multiple spritesheets in different frames.
function toSpriteSheetMaterial( material ) {
const frame = time;
// 2x2 tiles
const ssUV = spritesheetUV( vec2( 2, 2 ), uv(), frame );
material.colorNode = texture( material.map, ssUV );
return material;
}What I would like to really emphasize is the extend of a material, not the re-write of a material
three.js/src/nodes/accessors/MaterialNode.js
Line 122 in 7d25bc9
To extend a material with a behavior, it needs knowledge of the actual node to not break the built-in behavior
Here as a user that needs / want to extend the built-in behavior of the colorNode and keeping the rest of the built-in material exactly the same as it is supposed to behave :
var node;
const colorNode = material.color !== undefined ? color(material.color) : vec3();
if ( material.map && material.map.isTexture === true ) {
node = colorNode.mul( texture.sample( customUV ) );
} else {
node = colorNode;
}
Here is the trouble, for an extend and not a re-write your example would more likely be :
function toSpriteSheetMaterial( material ) {
var node;
const colorNode = material.color !== undefined ? color(material.color) : vec3();
if ( material.map && material.map.isTexture === true ) {
const ssUV = spritesheetUV( vec2( 2, 2 ), uv(), frame );
node = colorNode.mul( texture.sample( customUV ) );
} else {
node = colorNode;
}
material.colorNode = node
return material;
}
if we are using an instanced shader, then this will need to be read differently with vInstanceColor / instanceColorNode etc.. and this needs to be re-written in a few cases
Another example :
Rewriting an opacity node, without breaking the built-in behavior requires the knowledge of the actual opacityNode before re-writing it : the opacityNode read opacity and the alphamap if it exists
three.js/src/nodes/accessors/MaterialNode.js
Line 136 in 7d25bc9
it is real easy to build a custom node the way it's been currently built
What is harder is too build a material plugin that extend a built-in behavior, without breaking or missing any feature of the built-in, the way we were doing it in Legacy
I agree on the idea to have the uvNode.
It helps to change how uv() works in the rest of the nodes and not change all the other nodes using a customUV
Also, on a more architectural part I think nodeMaterial should be 100% node, and legacy code should be removed.
For example this part is convenient in legacy but should disapear if we want a node world in the futur :
if ( material.map && material.map.isTexture === true ) {
node = colorNode.mul( map.sample( uv() ) );
}
For legacy : i'll suggest to replace it in the constructor by the node equivalent directly and make the material.map as a getter / setter to the colorNode/mapNode ( to cut the colorNode in 2 and make more access point ) but not a real props if we want to keep this legacy things.
This way we can easily access anynode ( if they are named ) and override them / disconnect / reconnect to other node etc.. It make the extends of a material so easy and very close of what we have in unity / houdiny / blender.
Usually the replacement effort is minimal and can often be solved in a single line of code. In addition, it allows optimizations that would be impossible with the previous approach (modifying the existing process) which is more likely to cause problems. To replace UVs, it is necessary to follow the suggestions mentioned above.
When thinking about customizing TSL for Materials, we should think about extending nodes, and not extending materials through hacks as would be done with the previous approach. TSL Textures, is an example of this kind of extension. Once a Node is created, it is up to the user to choose where and how to use it.