"Summarize count() by <field>,bin_auto(_time)" statements do not visualize as different series
Opened this issue · 18 comments
While using the plugin version 0.3.0, I created a dashboard panel that showed the number of log messages logged at each severity level over time:
I used the following query to power this:
['global']
| where ['resource.env'] == "dev"
| summarize count() by ['severity'],bin_auto(_time)
I used the following transformation
to convert the result series from the format
to the format shown in the first screenshot.
After upgrading the Axiom data plugin to version 0.5.1, my dashboard shows a single count instead of individual counts per severity:
This seems like a regression. I expected that it would still show 1 line per severity. I have tried using Grafana transformations to get individual series again, but without success.
The dashboard used to demonstrate the bug is here:
series-bug-dashboard.json
Versions used:
Grafana v11.2.0 (2a88694fd3)
Axiom plugin: 0.5.1
hi @van-vliet, In version 5.0.0 we switched the returned data format, perhaps I missed something. I will take a look and get back to you.
@van-vliet Can you try to add a transformation that changes format from Long to Wide? That solved the issue for me for one of the charts. You can do that temporarily until I figure out the full solution.
@SollyzDev I am not sure what you are asking me to do.
I tried adding a type conversion transformation in Grafana. I see the following, but no options mention a long or wide format:
I also tried modifying my Axiom query to:
['global']
| where ['resource.env'] == "dev"
| summarize tolong(count()) by ['severity'],bin_auto(_time)
but that did not solve my issue either.
Can you provide an example of the change you performed that worked?
@van-vliet The transformation is called "Prepare time series". Add it, then choose "Multi-frame time series" like the screenshot below:
I will include a fix in the next version that will do this transformation automatically.
@dasfmi I see that this was potentially fixed here: #77
This is part of the release 0.5.2, which is available on Github.
Our installation pulls the plugin from here:
https://grafana.com/grafana/plugins/axiomhq-axiom-datasource/
When will version 0.5.2 be available there?
hi @jvv-trackunit , I have submitted the new version of the plugin to the Grafana marketplace but its still under review, once they approve it will be released. Usually it doesn't take all that time, will ping them.
Hi @dasfmi
Latest version on the Grafana marketplace is still 0.5.1. Is something blocking it?
@jvv-trackunit yes, the Grafana team required an update to the SDK to overcome a security risk the grafana-go-sdk version. I made the upgrade and the necessary changes and submitted a new v0.6.0 release. Waiting again for their approval.
@jvv-trackunit version 0.6.0 is out 👍
Hi @dasfmi
I tested my dashboard with the 0.6.0 plugin and the issue is still present. Can you reopen this?
@jvv-trackunit thats weird, you should at least see a different view now. Can you click inspect query on Grafana and show the query result?
Hi @dasfmi
Here is the query and response:
grafana-query.json
Grafana version 11.3.0
Axiom plugin version 0.6.0
hi @jvv-trackunit I just did a test and you are right, it doesn't automatically render multi-frame series results. A Fix is not straight forward since we now use different format for the query result. Although the solution I mentioned before works. On your chart add a new transformation, under "Prepare Timeseries" like the image below:
You should then be able to select multi-frame time series as a format:
I've implemented the workaround described here after upgrading to 0.6.0, but the resulting data appears to be incorrect. When running an identical summarize rate(1) by <field>, bin_auto(_time) query on both axiom.co and through the Grafana plugin, the data point values in Grafana are well over 10 times what axiom.co reports.
I don't see this kind of error if I do summarize rate(1) by bin_auto(_time).
EDIT: moved this to a separate issue
hi @jvv-trackunit I just did a test and you are right, it doesn't automatically render multi-frame series results. A Fix is not straight forward since we now use different format for the query result. Although the solution I mentioned before works. On your chart add a new transformation, under "Prepare Timeseries" like the image below:
You should then be able to select multi-frame time series as a format:
Adding the transformation works. EDIT: I did not verify if the counts are correct - only that the series show up. I have not checked whether I encounter the same issue as @tasgon describes above.
As a user, I'm not interested in updating all my existing dashboards created using earlier versions of the plugin to make them compatible with the latest version (0.6.0). I still consider this behavior a bug.
I'm hesitant to implement the workaround, since I expect having to remove it again if a future release fixes the issue.
Can you elaborate on the priority of this bug and if/when we might expect a fix?
hi @jvv-trackunit, I am sorry you are still facing this. This comes as a change to the format the query is returned from the API. In the older version it used to used to return the values for each groups, not it doesn't work the same way.
While debugging the plugin today I found out its failing to do the correct formation due to order of _time field. The simplest fix I found is to summarize by time first. This worked for me without any transformations:
I just noticed your query has bin_auto(_time) first, this still doesn't work for you? This could perhaps something in the result is causing this different behavior between my query and yours.
I would like to fix the bug for you, but haven't been able to put my hand on the problem.
Hi @dasfmi
My query puts bin_auto(_time) at the end. Moving it to the first grouping parameter seems to fix the issue. I think you have identified the root cause of the issue 💪
@jvv-trackunit It still bothers me why this happens, as it shouldn't behave like this, I will keep digging.