Embed query traces in the web UI
Kobzol opened this issue · 13 comments
Currently, if we want to open a query trace gathered from a self-profile, we need to go through ui.perfetto.dev. It takes considerable time to load and redirects to an external website.
It would be nice to integrate the visualization of the query trace directly within our website, e.g. using https://chromium.googlesource.com/catapult/+/HEAD/tracing/README.md or something similar.
On Chromium's issue list, it says that the browser's trace viewer will be replaced by Perfetto UI (maybe it provides better trace viewing features) . I also checked it takes time to redirect to the website, so should I work on it considering the 'replacement' point.
The main goal is to embed some visualization of traces within our web. I don't really care which visualization it is, as long as it works. If the used solution will be deprecated in the future, that doesn't have to concern us, as long as it will work in perf.RLO. And even if we had to replace it in the future, there will be some additional work that has to be done on the website that will be shared by all potential solutions, so it shouldn't be that difficult to replace the visualizer.
There are multiple open source trace visualizers, I would choose the one that's the easiest to embed.
Catapult traceviewer is not well documented, it only explains how to embed the traceviewer which I did, but it doesn't explain how will we create a tracefile (with .json.gz or .json.zip extension which is required) for a specified task, so I am not able to work on it further, can you suggest any documentation which explains it. (The closest I got was 'ftarce' for linux kernel).
rustc-perf already exports Chrome-compatible traces from rustc self-profile data, e.g. https://perf.rust-lang.org/perf/processed-self-profile?commit=b57a10c39d2a2da7389d029595d7fff6ac9cbf5a&benchmark=unicode-normalization-0.1.19-check&scenario=full&type=crox (taken from https://perf.rust-lang.org/detailed-query.html?commit=b57a10c39d2a2da7389d029595d7fff6ac9cbf5a&benchmark=unicode-normalization-0.1.19-check&scenario=full).
So if you can embed the visualizer, and point it to a URL address on perf.RLO, then that should be all (we might need to add a new endpoint that exports the data in the format that the visualiser expects).
As previously mentioned, several other projects using Catapult are migrating to Perfetto due to its unmaintained and deprecated status. There are already some bugs in Catapult.
They also discussed the trace packet proto, which is a protobuf format of trace data. It's noted to be somewhat smaller than JSON, potentially improving loading times.
https://perfetto.dev/docs/reference/trace-packet-proto
I'm curious about which way is better. I would like to test this format to evaluate how difficult it is to implement and how fast it loads. Based on the results, I will decide whether to use Catapult or keep using Perfetto with protobuf format export.
Additionally, if we decide to use Catapult, we might not be able to package everything into a single binary because Catapult uses Python and they don't distribute binaries.
(I'm not entirely sure about this, but there seems to be an option to process traces on the server side. I should look into this further.)
https://perfetto.dev/docs/visualization/large-traces
We can probably generate any format we like on the rustc-perf backend, so that shouldn't be a concern. The main goal is to enable displaying the traces within our own UI, without forcing the user to go to an external website (and especially to Perfetto, because it loads quite slowly and without any user indication of the loading duration, for security reasons).
Hmm, alright, then I'll try to integrate Catapult into rustc-perf!
because it loads quite slowly and without any user indication of the loading duration,
Would be very curious to see trace examples of traces showing this, happy to work on improvements or point you in the right direction on improvements to trace format.
or security reasons
Just as a note, note sure this is a valid point because the Perfetto UI is fully offline: there is no server side component. All processing just happens in the browser. Also the Perfetto UI is fully embeddable (it is embedded by several other projects including e.g. the Flutter Devtools Profiler) so if you're paranoid, you can build the UI from source and just embed the static artefacts in whatever other tool you want.
Would be very curious to see trace examples of traces showing this, happy to work on improvements or point you in the right direction on improvements to trace format.
You can try it here. Click on a row with a benchmark result and then on the "query trace" link. To clarify, we could probably add some loading indicator, but due to the fact that Perfetto requires the client to download the trace first, it's quite slow. We would really much more like to embed it.
Just as a note, note sure this is a valid point because the Perfetto UI is fully offline: there is no server side component. All processing just happens in the browser. Also the Perfetto UI is fully embeddable (it is embedded by several other projects including e.g. the Flutter Devtools Profiler) so if you're paranoid, you can build the UI from source and just embed the static artefacts in whatever other tool you want.
We would very much like to embed some query trace visualizer! But even though some of them look embeddable, we haven't yet found something that could be easily used "as a library" in Vue (although vendoring a directory full of HTML files and putting that inside an iframe or something like that is probably also an option, assuming that we could fully control how the data is loaded). Do you have some (source code) example of Perfetto being embedded?
You can try it here. Click on a row with a benchmark result and then on the "query trace" link. To clarify, we could probably add some loading indicator, but due to the fact that Perfetto requires the client to download the trace first, it's quite slow. We would really much more like to embed it.
I think what you are meaning by "embed" and what I am meaning by "embed" are two different things. When I say embed, I mean embed in some client side tool/website. I think you are treating it more as "run processing on server and stream data to the client as and when it needs to be displayed" (please do correct me if I'm misunderstanding here!). The reason I think this is because of your comment of "download the trace first" which is only possible to avoid if you have some server close to the data loading the trace remotely. If that's really the slowness you are looking to avoid, there's not much I could suggest to avoid that from the Perfetto perspective: we are, after all, a fully client side tool with no server component.
However, let me offer a couple of reasons as to why all the open source profilers go this routea) having a server side component means the user loading the traces needs to trust the server which is hosting the traces b) there's a lot of data which goes back and forth between the "backend" and "frontend": because of this, the client -> server latency massively impacts the UX of using the tool. Having both client and server in the browser means you pay the cost once but then after that, you have a very fast and responsive tool.
The way we make the load time good in Perfetto this is by reducing the size of the traces: you can see from the Go issue, they had a 280MB protobuf Perfetto trace which expanded to a 10+GB JSON trace: that's the sort of compression ratio you can expect and, if you're smart about use of protobuf format, you can go even better (the traces we load are usually in the 10-30MB range which take a couple of seconds to load).
Do you have some (source code) example of Perfetto being embedded?
I guess the Flutter devtools initial implementation PR (https://github.com/flutter/devtools/pull/4386/files) is a good starting point. Happy to guide more if you have specific questions. https://perfetto.dev/docs/visualization/deep-linking-to-perfetto-ui is the description of the official postMessage API that Perfetto supports and will be stable for the forseeable future as it's something we depend on very much as well.
Thanks for the detailed description! I think that we actually mean the same thing. While we do have a server that we trust and where we would like to do as much processing as possible, I didn't mean to do anything crazy like rendering the profile on the server and just sending the UI commands to it from the client. I formulated this badly; I know that the trace has to be downloaded to the client, but what I wanted to avoid was to require the user to open an external webpage/tab (which then also requires a security popup for accepting the trace), and also to avoid only opening the tab after the whole trace was downloaded (because the traces can be large).
The ideal scenario that I want to achieve is I think pretty simple:
- User clicks on a button
- The trace visualizer immediately opens directly on the same page, and immediately starts loading the trace directly from our server. If this can be "streamed" in the sense that data appears incrementally, great. If not, then at least some loading indicator in the visualizer would be nice.
That's pretty much it. I suppose that this should be possible with Perfetto, as you have shown with the Dart example (once again, thanks for that!). The iframe solution isn't the best of what I hoped for, but it seems workable. We will try this again in the future once we have a bit more time for it, and let you know how it went.
Thank you!
which then also requires a security popup for accepting the trace
Just as a note, this is a one time thing with ui.perfetto.dev: once you accept a trace from an origin once, we can store in localStorage that the user is happy to receive traces from that origin again.
and also to avoid only opening the tab after the whole trace was downloaded
Ack yes this is a bit harder to avoid but with some creative ideas I think is still possible. Still embedding in some other tool gives you more control over the UX which AFAIK you do want.
If this can be "streamed" in the sense that data appears incrementally, great.
This is a pipe dream we've wanted to build for like 6 years but it's a really hard thing to make end to end. Still some day, I hope we will get there :)
If not, then at least some loading indicator in the visualizer would be nice.
Perfetto does show a loading indicator of progress at the top about how much of the trace as a percentage + MB/s speed so the user does have some indication of UI not being totally frozen. We also put a bunch of effort into speeding up trace loads in the middle of this year so things might have improved since you last had a look.
I suppose that this should be possible with Perfetto
Yes can confirm that all you describe should be possible by embedding in another tool (or even creatively just with deep linking to ui.perfetto.dev but as I say, I think you want to control the UX more).
The iframe solution isn't the best of what I hoped for, but it seems workable
I think any other profiler you go with will be very similar, I'm not aware of anyone creating a JS library for visualizing traces (it's hard enough writing a timeline viewer as a tool let alone without making it a library :P).
I think any other profiler you go with will be very similar, I'm not aware of anyone creating a JS library for visualizing traces (it's hard enough writing a timeline viewer as a tool let alone without making it a library :P).
I think we found one that was split into components, but it was in React =D So no dice. Of course I do realize that it's incredibly hard to do this, and if it doesn't use the same framework as you then it's not super useful anyway (although embedding e.g. Vue components into a different framework is relatively easy).