Samsung/ONE-vscode

Actual tasks on 2023

Torrero opened this issue · 11 comments

Torrero commented

Dear @dayo09 @llFreetimell
I with @stamalakhov and @staddy would like to join to the ONE-vscode extensions development, can we discuss the actual tasks in this direction?

llFreetimell commented

Summary for 2022

Let's collect all the features which are developed until 2022 to plan actual tasks on 2023!

For each features, there are no restriction about refactoring, which means that we can even destroy and reconstruct from the beginning if it is the right way :)

And one more thing. Even if some improvement points are written, it maybe impossible to do so because reality is not yet fully considered! Just for reference to collect tasks :)

Official Features

These features are remarked as official and can be seen in release note.

▶️ JsonTracer

JsonTracer is for showing *.timeline.json file whose format is Tracing compatible.
The file is usually generated for profiling.

Current Status

Improvement Points

  • Embedding catapult instead of JsonTracer.
    • As current JsonTracer referenced catapult, embedding them has a benifit that we can fully support for trace event file.
    • UI is almost same.
  • Open with Chrome tracer
    • Chrome tracer is still maintained actively
    • If we just have correct formatted trace event file, we can always success to open the file.

Related Issues/Links

▶️ Mondrian

Mondrian is for showing *.tracealloc.json file whose format is internal.
For more details for the format, please refer to internal repository.
The file is usually generated for profiling.

Current Status

  • Display *.tracealloc.json with
    • Zoom in/out
    • Set sycle range
    • Detailed node information
    • Multiple segments

Improvement Points

  • Not yet discovered

Related Issues/Links

▶️ cfg Editor

There are so many options and arguments for writing *.cfg file which is used in ONE.
cfg Editor provides user-friendly and much easier GUI editor for *.cfg file.

Current Status

  • Support GUI for *.cfg file with
    • Enable/Disable each steps
    • Show all the options for each steps
    • Find file with file system dialog
    • Provide help message for some of options

Improvement Points

  • Support shortcuts
    • Ctrl + F5 : Compile cfg
    • Ctrl + ? : Automatically fill default options
    • (more)
  • Better UI
    • Current UI may not be best for users
  • Sync-up with multiple version of ONE
    • We may display for some options that they are only supported under specific version
    • If new feature is introduced in ONE, how can be sync-up effectively?
  • Backend dependency
    • For Codegen and Profile steps, they are related with backend.
    • Users may not know which value should be entered for various backend.

Related Issues/Links

  • None

▶️ ONE Explorer

Default vscode explorer is not friendly for ONE users.
ONE Explorer provides many useful features and UI for ONE users.

Current Status

  • Show related files
    • Base models(*.tflite, *.onnx, *.pb)
    • ONE configs(*.cfg)
    • Compiled models(*.circle, *.tvn)
    • Logs and profiling results (*.log, *.trace.json, *.tracealloc.json)
  • Open files with dedicated viewer/editors by clicking
    • *.cfg, *.circle, *.trace.json, *.tracealloc.json
  • Support run cfg
  • Support refresh on file system change

Improvement Points

  • Support viewers for .tflite/.onnx
  • React on command
    • To reduce redundant rebuilding of the tree
  • Support shortcuts
    • rename (F2)
    • delete (delete)
    • run cfg
  • Support drag & drop
  • Fix duplicated path error of multiple workspace

Related Issues/Links

▶️ ONE Toolchain (@jyoungyun)

Most of ONE users are hard to install and manage ONE related toolchains.
ONE Toolchain would decrease the resistance for using toolchains with convenient settings and UI.

Current Status

  • Support TRIX Toolchain
  • Support ONE Toolchain

Improvement Points

  • Support new version notification
  • Better UI
    • Support page view in Choose Compiler Toolchain
    • Toolchain mouse-over view

Related Issues/Links

  • None

▶️ ONE Device (@jyoungyun)

There are many devices and ways to run ONE toolchains.
ONE Device provides much intuitive device manager for ONE users.

Current Status

  • Show runnable devices on local machine
    • Requires internal vsix package

Improvement Points

  • Support real target H/W
  • Run backend binary on remote machine with
    • Real target H/W
    • S/W for running binary
  • Compare and show the running results
  • Profiling

Related Issues/Links

  • #457
  • Internal repository

Unofficial Features

These features are included in the release package, but not remarked as official.
As a result, we can execute them with release package but just remember that there can be some bugs or unresolved issues.
When they are ready to be official, they would be official features in the future!

⏯️ Metadata Viewer

Current Status

TBA

Improvement Points

TBA

Related Issues/Links

⏯️ Metadata Manager

Current Status

TBA

Improvement Points

TBA

Related Issues/Links

⏯️ circle Viewer

Before ONE-vscode is introduced, netron was used for *.circle file viewer.
Now, circle Viewer embedded netron into ONE-vscode and provides a viewer for *.circle files.

Current Status

  • Show *.circle file with netron UI.

Improvement Points

  • Not yet discovered

Related Issues/Links

  • None

⏯️ PartEditor

Editor for *.part file which is required by circle-partitioner in ONE.

Current Status

  • Edit *.part file as GUI

Improvement Points

  • Unconfirmed

Related Issues/Links

⏯️ circle Editor

There was no editor for *.circle file, only viewer was.
circle Editor enables editing circle file with familiar circle file viewer UI.

Current Status

  • Able to edit following things in *.circle file
    • Attributes
    • Inputs/Outputs
    • Weight values
    • Shape
    • Type

Improvement Points

  • Edit raw file as JSON
    • Advanced user may want to edit minutely model file
    • There maybe unseen information at the editor UI
  • Performance
    • For now, whole model file is overwritten regardless of change size
    • Unlike viewer, editor requires much computing resources

Related Issues/Links

⏯️ visquv

As a result of visq, *.visq.json is generated which includes quantization error metrics.
visquv shows the file as GUI.

Current Status

  • Show *.visq.json as circle file viewer GUI
    • Related circle model path is included in the *.visq.json file

Improvement Points

  • Support *.visq.json.dot

Related Issues/Links

Feature Requests

These features are not developed at all.
However, they are either good to have or must to have in the near future :)

⏸️ workflow Editor

Motivation

ONE has introduced workflow in onecc but ONE-vscode is not yet supporting it as GUI.

Candidate Features

  • Show *.workflow.json with good GUI
  • Support shortcuts
    • Ctrl + F5 : Compile cfg
    • Ctrl + ? : Automatically fill default options
    • (more)

Related Issues/Links

⏸️ Quantization Utility

Motivation

cfg has Quantize tab, but creating configuration files are not familiar to most of users.
In addition, applying quantization itself is not that easy for newbies.
If there is an awesome tool all about quantization, many users who wanted to quantize easily maybe happy!

Candidate Features

  • Integrate all of quantization tools into one
    • Create *.qconf.json easily
    • Run circle-quantize with GUI
    • Support mixed precision quantization with GUI
    • (more)

Related Issues/Links

⏸️ Integrated circle File Viewer/Editor

Motivation

There are quite a lot of viewer/editor for circle file, such as circle Viewer, PartEditor, circleEditor, VisqViewer, and so on.
However, they have all the same netron UI.
It implies that they have following problems.

  • Same(feature is diferent but UI is same) viewer/editor
    • If there are more than 2 viewers, users maybe confused due to same appearance.
    • Users should open many same viewer/editors when they need to do something different.
  • Many duplicate codes

Candidate Features

  • Integrated circle file viewer/editors into one

Infra/Etc

These are not features but must be maintained for ONE-vscode development and release.

▶️ code-marketplace (@jyoungyun)

  • When new extension is released, it should be uploaded to marketplace.
  • Internal extension is needed to be distributed but cannot be uploaded Public marketplace.

▶️ SE

  • Potential defects are better to be removed
  • There should be no license & copyright issues
  • Tests should be enough to ensure the correctness
  • 👍5
llFreetimell commented

@Torrero, @stamalakhov, @staddy

All the candidate tasks are collected :)

staddy commented

We have also discussed the feature of weight visualization. It will help to detect model issues easily and speed up debugging. Then intermediate tensors, errors, etc. visualization may also be implemented.
@dayo09 @llFreetimell, what do you think about this feature?

jyoungyun commented

what do you think about this feature?

This feature can be introduced in ONE-vscode. Welcome!

PartEditor was implemented by @seanshpark. He has a lot of knowledge about netron and UI implementation in VSCode extension. He can help you with what you're trying to do. /cc @seanshpark

First of all, please make an issue of what you want to do for weight visualization.
And let's continue the discussion there.

  • 👍1
jyoungyun commented

@Samsung/one-vscode

We need to discuss how to manage and proceed with this project in the future.

As far as I know, the current ONE-vscode project has only tow members @dayo09 and @jyoungyun.
Unfortunately, @llFreetimell can no longer participate. He goes to study for two years.
This means that no one has merge rights in ONE-vscode. (In fact, @llFreetimell does not have the role of merging PR, but he did so because no one has the permission to merge. Thank you.)
And sometimes @seanshpark and @mhs4670go helped us merge our PR. (Thank you!)

In ONE-vscode, AMPQ or above items will be developed this year. (Who? 👀)
How should we proceed with this project in the future?

  • 👍2
seanshpark commented

In ONE-vscode, AMPQ or above items will be developed this year. (Who? 👀)

I think I could do some reviews related with UI and circle viewer control applications :)

  • 👍4
stamalakhov commented

@seanshpark @dayo09 @jyoungyun
BTW is anyone interested in providing/viewing metadata for compiler products?

jyoungyun commented

I'm going to implement only toolchain related things in ONE-vscode this year. :)

  • 👍1
dayo09 commented

@stamalakhov It was planned and drafted last year, but not merged. If you are interested, you can go on with that. I am currently not planned, too

  • 👍1
stamalakhov commented

@stamalakhov It was planned and drafted last year, but not merged. If you are interested, you can go on with that. I am currently not planned, too

ok. Thank you.

  • 😄1
stamalakhov commented

I'm going to implement only toolchain related things in ONE-vscode this year. :)

ok. Thank you.

  • 😄1