[UX] Memory management visualization

[ai-moiseev]

Let's support memory measurement visualization tools.
Parent issue: #309

This feature depends on ONE frontend support and also on backend implementations.

Required steps

Required external steps:

• Provide unified format for memory allocation description file (*.tracealloc.json)
• Implement memory allocation visualization support in ONE Samsung/ONE#8723
• Provide support in backend (UPDATE: support was added in external repo at 2022-04-03)

ONE-vscode support:

• Provide a checkmark option for .tracealloc.json generation during a compilation
• Implement UI for allocation-over-time display (with support for specification of cycle range)
  • Common UX functionality (memory zone viewer using custom editor, view scaling) #483
  • Ability to select active cycle range (to improve view performance for large models) #748
  • Hover tooltip with additional zone info (origin, size, offset, lifetime) [implemented in status bar info on hover]

Additional UX improvements:

• Dynamic grid and rulers
• Fit-to-window button

[hyunsik-yoon]

Thanks for this work.

Provide support in backends

Can you also discuss in the repo of backends you're considering?

[ai-moiseev]

Can you also discuss in the repo of backends you're considering?

Sure, there's already a discussion in backend repository, I've added this item for external progress status tracking :)

[dayo09]

@ai-moiseev

I just checked the spawned issue in ONE repository, it seems you are planning to create circle memory visualization.
I hope you note that ONE-vscode targets mostly on model files for npu-backend. We do plan on extensive support on other file formats(circle, ...) but our initial goal this year is to support npu-target files.

[ai-moiseev]

@dayo09 The idea was to introduce common format in ONE to provide support for more than one backend, but current focus is on NPU target. I've recently added support for export in JSON format in backend's compile command, and I plan to add this new format specification to markdown documentation.

This JSON format comes as an update on mondrian CSV files with support for versioning and additional metadata, so this effort is part of #309 (C Support view for mondrian csv file).

[hyunsik-yoon]

@ai-moiseev
How about discussing UI you're in mind?
E.g., sharing some sketch could help.

[ai-moiseev]

@hyunsik-yoon Here's UI concept which I currently have in mind:
It closely follows output of existing mondrian utility, but as addition it provides a way to interactively select cycle range, scroll to begin or end of the model, show additional info on mouse hover and switch between memory segments.

[ai-moiseev]

Plugin feedback notes from @hyunsik-yoon (link: #483 (comment))

1. Some people may not know what this json file is. So , when they click the json file, how about showing some title like Memory map over time and some description on top of graph?

2. After opening json file, right-bottom area shows memory, cycle, and segment. I guess memory here means total memory required through whole times? If so, how about using Total memory label?

3. When I select segment 2, memory is 16384 and I am not sure why. From the graph, it has one segment of 8K size. So would 8K be right for memory? (So does segment 1)

4. I guess X axis would be time (cycle??) and Y axis would be offset but I am not sure. How about showing the meaning of X and Y axis?

5. When I maximize the size of vscode to full screen and click the json file, initial graph is shown in small top left area. Maybe when user clicks json, how about fitting the graph to screen size?

6. Some button like 'Fit to screen size' would provide good usability.

7. In the json file, segment 1 has 24 cycles and segment 2 has 32 cycles. When I enlarge segment 1 to fit all cycle in width of screen and then select segment 2, segment 2 also shows 24 cycles in screen. So I fit 32 cycles of segment 2 to the width of screen. Then I select segment 1 again and found that graph is now shrunken and I had to enlarge again. I hope zoom level of each segment should be kept separately.

8. Zooming granularity of '+' and '-' seems kinda large, I guess.