Stratus3D/eflambe

[ENHANCEMENTS] Scaling widths by metrics + callstack categorisation/grouping

Opened this issue · 4 comments

@Stratus3D I'm really enjoying eflambé and i'm wondering if it's possible to add these two innovations from flame_prof:

High scheduler load

The reductions heatmap and corresponding flamegraph below show what callstacks contributed most to schedular load in the example profile. Colour saturation in the heatmap and widths in the flamegraph are scaled to reductions. Scaling flamegraph widths by metrics measured during sampling is the first of flame_prof's two main innovations.

Crucially, in the flamegraph, the callstacks are categorised by the process status of upto 100 processes running that particular code (the example profile auto-selected the top 100.) Such callstack categorisation/grouping is flame_prof's other main innovation. It has proven extremely useful especially in identifying code causing concurrency bottlenecks or memory issues.

Armed with them, eflambé could be the best profiling tool for Erlang.

eFlambé doesn't do heatmaps, it does flamegraphs/callstacks. Aside from being simpler, it allows us to use different flamegraph viewers interchangeably and only have to worry about the file formatting differences. I don't plan on ever changing this and don't have the desire to support heatmaps like flame_prof does.

eFlambé only profiles function calls and uses runtime to set the widths in the flamegraph. We can definitely add an option to use runtime, reductions, or other metrics to set the widths in the flamegraphs.

Scaling flamegraph widths by metrics measured during sampling is the first of flame_prof's two main innovations.

Can you provide a list of metrics that frame_prof can use to set the widths in flamegraphs?

I'm thinking of adding a {scale_by, metric()} option to the list of options that can be passed to the eflambe functions where metric() is one of runtime, reductions, etc... Thoughts?

@Stratus3D ok for heatmaps.
Regarding metrics (random order):

  1. reductions
  2. garbage collection (heap block sizes, etc)
  3. group callstacks by the memory used by those processes when the calls were being made.
  4. bin_vheap_size
  5. process message queue lengths
  6. message buffer sizes
  7. stack memory

@Zabrane does flame_prof implement all these? And if so, can you link to the relevant source code? Thanks!

Some of these metrics seem easier to implement than others, but I haven't started on this yet.

Note to self: look at the source code for https://github.com/kzemek/erlang_pmp