brendan-duncan/wgsl_reflect

Get unused members

wilbennett opened this issue · 10 comments

wilbennett commented

Hello,

I was getting ready to write something similar and came across your repo. I took a quick look and it doesn't seem to have a few features I need. Sorry if I missed them.

  • List of structs that are not referenced
  • Starting and ending line number of each member (structs and functions)
  • List of custom functions referenced by a function

My use case:

I build up shaders by including snippets. This may cause the shader code to be unnecessarily large because it includes code that is not being used. I would like to remove all the unused elements after the includes have been processed.

Example:

#include common_structs
#include vec2_utils
...

@vertex
fn vs...

After expansion of the includes, I would run reflect and identify any structs and functions that are unused by the vertex shader. I would then remove them so the final shader code is smaller.

Thanks,
Wil

brendan-duncan commented

I think those features are doable. I'll try to get to them sometime soon. Ping me if I forget, work has been busy.

wilbennett commented

Thanks! No worries, I know how it is - I'm pretty busy now myself.

brendan-duncan commented

Chipping away at it while it's fresh in my mind, and waiting for my work project to compile.

I added startLine and endLine to StructInfo and FunctionInfo, and I added "functions" to WgslReflect, which is the list of all functions (including entry functions).

I'll add struct and function tracing next, probably adding a member to StructInfo and FunctionInfo along the lines of "inUse: boolean", and I can add a list of all the custom functions called from an entry function (directly and indirectly). But if the function inUse is false, that means the function wasn't called, so it's questionable if the list is necessary.

Here's the code I have for the unit test for line numbers,

const reflect = new WgslReflect(`
      struct A {
        a: u32,
      };
      struct B {
        b: u32,
      };
      struct C {
        c: u32,
      };
      @group(0) @binding(0) var<uniform> foo : A;
      @vertex
      fn vertex_main() -> void {
        let b = B(1);
}`);
    test.equals(reflect.structs.length, 3);
    
    test.equals(reflect.structs[0].startLine, 2);
    test.equals(reflect.structs[0].endLine, 4);

    test.equals(reflect.structs[1].startLine, 5);
    test.equals(reflect.structs[1].endLine, 7);

    test.equals(reflect.structs[2].startLine, 8);
    test.equals(reflect.structs[2].endLine, 10);

    test.equals(reflect.entry.vertex.length, 1);
    test.equals(reflect.entry.vertex[0].startLine, 13);
    test.equals(reflect.entry.vertex[0].endLine, 15);

    test.equals(reflect.functions.length, 1);
    test.equals(reflect.functions[0].name, "vertex_main");
  • ❤️1
brendan-duncan commented

Squeezed in one more update for the night: added inUse and calls fields to FunctionInfo, as in:

FunctionInfo {
  inUse: boolean;
  calls: Set<FunctionInfo>;
}

inUse will be set to true if it's called either directly or indirectly from an entry function.
calls is the set of all custom functions called either directly or indirectly from the function.

As an example, the unit test has been updated to

const reflect = new WgslReflect(`
      struct A {
        a: u32,
      };
      struct B {
        b: u32,
      };
      struct C {
        c: u32,
      };
      fn notInUse() -> void { }
      fn inUse2() -> void { }
      fn inUse() -> void { inUse2(); }
      @group(0) @binding(0) var<uniform> foo : A;
      @vertex
      fn vertex_main() -> void {
        let b = B(1);
        inUse();
      }`);
    test.equals(reflect.structs.length, 3);
    
    test.equals(reflect.structs[0].startLine, 2);
    test.equals(reflect.structs[0].endLine, 4);

    test.equals(reflect.structs[1].startLine, 5);
    test.equals(reflect.structs[1].endLine, 7);

    test.equals(reflect.structs[2].startLine, 8);
    test.equals(reflect.structs[2].endLine, 10);

    test.equals(reflect.entry.vertex.length, 1);
    test.equals(reflect.entry.vertex[0].startLine, 16);
    test.equals(reflect.entry.vertex[0].endLine, 19);

    test.equals(reflect.functions.length, 4);
    test.equals(reflect.functions[0].name, "notInUse");
    test.equals(reflect.functions[0].inUse, false, "notInUse");
    test.equals(reflect.functions[0].calls.size, 0, "notInUse calls");
    test.equals(reflect.functions[1].name, "inUse2");
    test.equals(reflect.functions[1].inUse, true, "inUse2");
    test.equals(reflect.functions[1].calls.size, 0, "inUse2 calls");
    test.equals(reflect.functions[2].name, "inUse");
    test.equals(reflect.functions[2].inUse, true, "inUse");
    test.equals(reflect.functions[2].calls.size, 1, "inUse calls"); // inUse2
    test.equals(reflect.functions[3].name, "vertex_main");
    test.equals(reflect.functions[3].inUse, true, "vertex_main");
    test.equals(reflect.functions[3].calls.size, 2, "vertex_main calls"); // inUse, inUse2

So I think the remaining requested feature is to add inUse to structs to see if they have been used, as a binding type or within a used function.

Unless I completely missed the mark on what was asked. That happens sometimes.

  • 👍1
brendan-duncan commented

Ok, added inUse to StructInfo for struct tracking. Guess I got to this quicker than I was expecting, minus extensive testing. I think I got to all of your requests, other than any feedback.

Here is the example unit test including struct tracking,

    const reflect = new WgslReflect(`
      struct A {
        a: u32,
      };
      struct B {
        b: u32,
      };
      struct C {
        c: u32,
      };
      fn notInUse() -> void { }
      fn inUse2() -> void {
        let b = B(1);
      }
      fn inUse() -> void { inUse2(); }
      @group(0) @binding(0) var<uniform> foo : A;
      @vertex
      fn vertex_main() -> void {
        inUse();
      }`);
    test.equals(reflect.structs.length, 3);
    
    test.equals(reflect.structs[0].startLine, 2);
    test.equals(reflect.structs[0].endLine, 4);

    test.equals(reflect.structs[1].startLine, 5);
    test.equals(reflect.structs[1].endLine, 7);

    test.equals(reflect.structs[2].startLine, 8);
    test.equals(reflect.structs[2].endLine, 10);

    test.equals(reflect.entry.vertex.length, 1);
    test.equals(reflect.entry.vertex[0].startLine, 18);
    test.equals(reflect.entry.vertex[0].endLine, 20);

    test.equals(reflect.structs[0].inUse, true, "A inUse"); // A, used by uniform
    test.equals(reflect.structs[1].inUse, true, "B inUse"); // B, used by inUse2
    test.equals(reflect.structs[2].inUse, false, "C inUse"); // C is not used

    test.equals(reflect.functions.length, 4);
    test.equals(reflect.functions[0].name, "notInUse");
    test.equals(reflect.functions[0].inUse, false, "notInUse");
    test.equals(reflect.functions[0].calls.size, 0, "notInUse calls");
    test.equals(reflect.functions[1].name, "inUse2");
    test.equals(reflect.functions[1].inUse, true, "inUse2");
    test.equals(reflect.functions[1].calls.size, 0, "inUse2 calls");
    test.equals(reflect.functions[2].name, "inUse");
    test.equals(reflect.functions[2].inUse, true, "inUse");
    test.equals(reflect.functions[2].calls.size, 1, "inUse calls"); // inUse2
    test.equals(reflect.functions[3].name, "vertex_main");
    test.equals(reflect.functions[3].inUse, true, "vertex_main");
    test.equals(reflect.functions[3].calls.size, 2, "vertex_main calls"); // inUse, inUse2
  • ❤️1
wilbennett commented

Wow! That is awesome! Thanks for your quick response. I do agree that with the inUse flag, the list of functions aren't strictly necessary. It can be good for debugging though, for example displaying a graph to get an overview of the flow in order to tweak dependencies.

Everything looks good. The one thing I would change is, if keeping the list of functions, the list should only contain those functions called directly. My reasoning is you have all the direct references and can "walk" the list to get all indirect ones, if needed - makes creating a graph pretty simple. If they are all in the list, you can't tell what is direct from indirect. Memory usage will also be less because the same information won't be repeated at multiple levels. What do you think?

brendan-duncan commented

Yeah, that makes sense from a graph perspective, and easy enough to change. I'll clean that up shortly.

I also finally got around to putting the library up on NPM, make things a bit easier to maintain. Better late than never, only took me two years to get to that 🙂

  • 👍1
  • 😄1
brendan-duncan commented

K, I think I got that fixed up, FunctionInfo.calls should only list the direct custom function calls from that function. I pushed the publish version. Trying to get to all this before I get swamped again with work and family stuff again in the coming couple of weeks.

  • ❤️1
wilbennett commented

Oh BTW, I came across your repo via webgpufundamentals.org and their reference to webgpu-utils which uses this library. I believe that's by user greggman.

brendan-duncan commented

Gregg is great, he was an early user of the library for his great tool. I had originally written this library for a project I ended up not working on anymore as I got too busy, https://loki3d.com. I was intending on migrating it to WebGPU but other things got in the way. Now I've been using it for my other project, a WebGPU debugger https://github.com/brendan-duncan/webgpu_inspector.

I'll be seeing Gregg tomorrow (March 20th), we're both presenting at https://www.khronos.org/events/webgl-webgpu-meetup-GDC-2024.

  • ❤️1