[Chart] Create Andon chart

Question

[Chart] Create Andon chart

Closed this issue 3 years ago · 8 comments

Motivation

We would like to create an andon/traffic light chart.

This visualisation is a great tool to get a KPI status or a situation at a glance: good, bad, to monitor, action requested, etc. It can be used with qualitative data too:

This makes it useful with qualitative metrics, and for quantitative ones when knowing the scale (as it is the case with gauge charts) isn't relevant.

Proposed Functionality

The chart would be constituted of one (or many?) item. Each item is constituted of:

a background block, coloured based on the value,
(optional) a prefix — generic to every item,
(optional?) a (possibly formatted) value — specific to each item,
(optional) a suffix — generic to every item,
(optional) a title (or label) — specific to each item.

It is expected for the block to be conditionally coloured, based on some predefined rules.

Alternative

We could build upon EChart heatmap, but this seems a bit of a stretch.

Help requested

We would like to refine the scope of this visualisation type, in order to develop something generic and useful enough to be included as part of Superset default chart suite.

I believe we need your inputs to find the right balance between making it generic and making it customizable. Plus I would appreciate feedback on what is the best approach for the user to specify how the block should be coloured.

Expected input:
I can imagine two approaches:
1. return (title, value) pairs only: the advantage is to retrieve relevant data only (lower load and simple query on the data source) and to separate the content from the form (as those preparing the query might not be those building the dashboard). Yet this makes harder to implement (technically — even if this shouldn't matter — but also in the case of more complex/nested conditional statements) and having logic to be implemented via the GUI (i.e. the chart settings) doesn't appeal to my dev mindset for re-usability, scalability, versioning, etc. One could also argue that this would avoid hard-coding colours in the query, to cope with various pallets… yet if the chart settings map to “relative colours” (e.g. “if value ≥ 42 AND value < 82 then 'first colour of the palette'”), this could lead to possibly dangerous situations where one swap green and red when updating a given palette (I know it sounds a bit far fetched, but still).
2. return (title, value, colour): the advantage is to move the logic into the query, what makes it easier to implement on our side, but also gives more freedom and power to the user (e.g. by implementing their own colour assignment function on db side — what could then be easily re-used over multiple charts). When using a cache, this avoids heavier rendering load at runtime. Yet not every user may be SQL-fluent enough to implement it. Plus some might argue that what the data are should be separated from how they should be rendered.
- Which approach should we favour?
  This can be answered in absolute terms, but also to be consistent with the UX in the other default charts.
- Should the “value” always represent the KPI's value?
  Instead of showing the KPI value itself, one might want to display a “message” instead (I can mainly think of symbols like ✔, ✗, etc.). I believe that these messages would be redundant with the block colour conditional formatting. Should we then move the “message” logic into the graph setting as well?
Count of items:
- Do we want allow for only one item per chart (cf. BigNumber), or (possibly) multiple ones?
  Restricting the chart to display only one item would be easier to design and implement. But I can imagine use cases when having multiple items could be useful — either because one doesn't know how many will be beforehand (e.g. if using a GROUP BY query), or if there would be too many of them, so creating each instance would be tedious and not DRY.
- Would the count of items be dynamic (based on the number of records in query response) or fixed (as chart parameter)?
  Again, having a fixed count of items makes it easier to implement. Yet the visualisation is likely to be suboptimal if the query returns more (or less) records than expected. In the other hand, having a dynamic count could lead to weird situations, should the query returns 100s (or even 10s) of records?
- How to position multiple items relatively to each others?
  A flexbox-like behaviour (with possibly dynamically resizeable children elements) seems indicated. But what should happen if there are too many items?
Shape:
- Should we allow users to select the shape on the item?
  Frankly, I wouldn't want our users to start using the whole range of MS Powerpoint-like shapes… yet relying on colour only (especially those from pallets that don't consider colour blindness) isn't the best for accessibility.
- Should this shape be “conditional formatting-able”?
  The risk is for dashboards to start looking like this. But making colour and shape redundant could be a great way to attract attention, especially for this kind of “overall KPI” data viz type.
Rendering:
- Should we display the “non-activated” options?
  On a traffic light or traditional andon stack light, the information is conveyed not only by the colour of the lit up lamp, but also by the positioning of the lit up light relative to the other. The benefit is that you can get a sense of the magnitude (is there an emerald step after red or are we in worst-case mode already?). Yet this takes a lot of “ink” for very low-value added elements. Plus this require to order the various scenario, what might not always make sense (is white greater or lower than blue?).
Name:
- Should we name this chart andon?
  Andon is likely not a household name. Plus people familiar with it might expect a stack-light type of chart. At the same time, “traffic light” might be misleading, especially if we depart from red-amber-green scenarii. Suggestions welcome!

Answer 1 · 2021-03-27T01:15:10.000Z

@EBoisseauSierra thanks for the proposal! Andon is a great chart to have in Superset. the only question before getting into the details is whether we should make it a Superset default chart or a viz plug-in. If we make it a default chart, we will need to replace Big number chart. Can we add trend line to it later on for consistency?

a background block, coloured based on the value
Automatic setting color based on the set threshold? or purely manual color setting?

@suddjian @rusackas

Answer 2 · 2021-03-27T08:43:37.000Z

whether we should make it a Superset default chart or a viz plug-in

Oh, right, sorry I wasn't clear. I meant a viz plug-in, but design it in a way that would make it useful/generic enough to possibly be merged and offered as part of Superset core suite of charts.

If we make it a default chart, we will need to replace Big number chart.

I'm not sure I would agree, as while these charts are similar in the sense they all help visualise one single value, they — imho — all have their value as they help answering different questions:

BigNumber: what is the KPI value?
BigNumber with trend-line: what is the KPI value? + how has it evolved recently?
Gauge: what is the KPI (absolute) value? + how does it compare relative to a range of possibility?
Gauge with intervals: what is the KPI value? + how does it compare relative to a range of possibility? + how should I interpret KPI value?
Andon: how should I interpret KPI value?
Andon with value: what is the KPI value? + how should I interpret KPI value?

… or in other words:

Chart	Absolute value	Relative value	Interpretation
BigNumber	✓	✗	✗
BigNumber w/ trendline	✓	✓ _{(relative to previous values)}	✗
Gauge	✓	✓ _{(relative to predefined range)}	✗
Gauge w/ intervals	✓	✓ _{(relative to predefined range)}	✓
Andon	✗	✗	✓
Andon w/ value	✓	✗	✓

Answer 3 · 2021-03-29T13:41:42.000Z

Of course, we wouldn't want to reinvent the wheel — and ECharts are currently the way to go. They don't have an Andon chart out of the box, yet adapting their scatter chart seems it could be a good way to go: see here and there.

Answer 4 · 2021-03-30T17:15:53.000Z

Had a meeting with @EBoisseauSierra.
We will be looking into different options

If we are able to implement conditional formatting(coloring), formatting timestamps, easily on the existing big number chart, let's make incremental changes on Big number and Big number with Trend
Since Aton will not be able to replace Big number and Big number with trend, we may wanna look for a different library, maybe d3.js that offers all the features mentioned above + can reach feature parity out-of-the-box. Then we can swap the Big number charts
Build Andon using the dynamic viz plug-in system apache/superset#10288

cc @villebro @ktmud @zhaoyongjie @suddjian

Answer 5 · 2021-03-30T18:43:57.000Z

@EBoisseauSierra Thank you for the detailed proposal. I've seen a lot of dashboards with multiple big number charts, so this could be a great extension of these use cases.

Here are my 2 cents on your open questions:

Return (title, value) vs return (title, value, colour): I'd recommend managing colors in the frontend to be consistent with other charts. Instead of setting each GROUP as an andon item, maybe we can treat each metric as an item instead. Users can add multiple metrics and the title/suffix would just be metric name. Colors (and conditional formatting) can be mapped to each metric using ColumnConfigControl we just introduced for table chart.
Should the “value” always represent the KPI's value?: yes, it'd be a metric in Superset's context and could return either numeric or string values.
Do we want allow for only one item per chart (cf. BigNumber), or (possibly) multiple ones? I think you can allow for multiple items by treating each item as a metric. It'd be much more convenient than creating multiple charts and adding them to Dashboard separately. The number of items can be dynamic based on how many metrics users added.
How to position multiple items relatively to each others? What should happen if there are too many items? They can be easily arranged horizontally or vertically with flex box (you can add a layout: 'horizontal' | 'vertical' chart config). Users have control on how many items to add based on how many metrics they add.
Should we allow users to select the shape on the item? Maybe not in the first iteration. But we could provide a simple style configuration such as borderRadius of the boxes.
Should this shape be “conditional formatting-able”? Let's not worry about this in the first iteration. Colors should be good enough for most cases.
Should we display the “non-activated” options? yes, each item is just a metric. If a metric is added in the list, display it. Give null values a special style.
Should we name this chart andon? This is the first time I've heard of andon so I couldn't guess what it is when I first saw it. Maybe "Stack light" is good enough?

Answer 6 · 2021-04-08T11:28:33.000Z

We would prefer not to reinvent the wheel, and so use components from a readily available chart library.

npm trends lists most popular JS charting libraries by count of download:

This is consistent with stackshare best charting lib.

Thanks to review by @krsnik93 we know that none of them has a native “andon”-like chart.

The closest we get is in plotly's Indicator — which looks like a replica of Superset's legacy BigNumber (yet without the trendline).
But it looks like (cf. here and there) we could also piggyback on echarts, what would have some advantages as to not introduce a new dependency and keep a consistent look-and-feel across the different charts.

Answer 7 · 2021-04-08T17:15:20.000Z

IMO this chart is simple enough to be easily implemented with regular DOM elements and a flexbox layout. But obviously it's up to you if you are just going to develop this vis as an optional plugin.

Answer 8 · 2021-04-20T16:04:05.000Z

Implemented here: https://github.com/krsnik93/plugin-chart-status-indicator.