Let's implement weight visualization feature

[staddy]

What?

Let's implement weight visualization feature.

Why?

It will help to detect model issues easily and speed up debugging.

How?

@seanshpark, @jyoungyun, @dayo09, @stamalakhov, let's discuss it here.

[staddy]

2D weights can be visualized as a heatmap that appears, for example, after clicking on a node in CircleViewer.
For 3+D weights the respective axes can be selected.
We may also discuss 3D visualization.

[seanshpark]

after clicking on a node in CircleViewer.

1. can you please share some sketch how it appears ?
2. can you please roughly describe user interaction and response of vscode?

[staddy]

@seanshpark,
I'm assuming we can handle node selection changed event from Netron (do you know if it's possible?).
So if the feature is enabled and a compatible node is selected a subwindow becomes visible.
The subwindow contains:

• minimum and maximum weight values, weights shape, etc.
• (if there are >2 axes with length > 1) a GUI to select 2 axes that should be used for heatmaps.
• heatmap image(s) and a GUI to switch between different heatmaps.

Here is an example of existing software, that inspired us originally:

I will provide my GUI sketch as soon as I get familiar enough with ONE-vscode.

[seanshpark]

Thanks for the explanation! :)

from Netron (do you know if it's possible?).

I recommend to use CircleGraph which I modified as a common control(?) for circle graph view.

In media/CircleGraph/index.js, this handler is called when selection is changed.
You can follow caller or callee and understand how it works.
If you need more assist, please ping :)

  /**
   * @brief called from view.js for handle view events
   */
  onView(event) {
    if (event === "selection") {
      // there was a selection change and view requested to apply this
      let v = this._view.getSelection();
      vscode.postMessage({
        command: "selection",
        names: v.names,
        tensors: v.tensors,
      });
    }
  }

[staddy]

@seanshpark, thank you.
I also had a question about code in 'external' directory, can we modify it?

[seanshpark]

I also had a question about code in 'external' directory, can we modify it?

files in external folder is copied as-is from upstream, without modification and without format check. purpose of this management is to make upgrade(change apply) from upstream a bit easier in IDE.

if you like to modify it, please make a copy to it's parent, modify caller code (like index.html).
parts of the code may be modified by the format checker, but resulting be harder to read.
you can add some directives like // clang-format off to prevent it; you can refer from existing codes in CircleGraph folder, but there is no strict rules for this.

[staddy]

@seanshpark, I've implemented basic visualization draft:
staddy@f2627b9

Planning to refactor the code and implement the following features:

• Axes selector
• Value selector
• Value tooltip

[staddy]

@seanshpark, all planned items are implemented:
staddy@82bb390
staddy@e466802
staddy@b35c070

Should I create a pull request now?
Also is it ok to have standalone functions there?

[seanshpark]

Should I create a pull request now?

PR with one context per changes with comments explaining the purpose
would help review to understand the codes :)
See below comment.

Also is it ok to have standalone functions there?

I can't catch the meaning of standalone functions and there. Anyway we can see with the PRs.

[seanshpark]

I don't have knowledge to understand the your new controls: axes selector and values selector,
and how they affect the graph rendering.
Some technical documents included in the change, not just codes, may help understand others
and to improve this functionality from others in the future.

[seanshpark]

Should I create a pull request now?

Please post a working draft PR and let others see all the codes and try out before landing.

• I don't fully understand the changes so below comments may be incorrect
• I assume your current changes are for property side-bar window only.
• I don't want you to change default behavior of the property side-bar of the graph view; you should add a flag to enable/disable it
• Working draft means from the draft code, anyone can try out the new feature, but functionality may be imperfect with bugs.

[seanshpark]

I think it would help if you can describe detailed UI to show weight visualization of the circle model.