Please improve response times
Opened this issue · 14 comments
As developers we depend on the Grafana plugin repository for discoverability of our plugins. Other projects like e.g. node-red have found less restrictive plugin mechanisms by utilising e.g. npm and keywords but for the time being it is this repository.
Review and merge times are abyssal:
I'm kindly asking Grafana to improve response times for reviewing and merging plugins. While I understand that this might not be highest priority our plugins contribute to the success of Grafana and do deserve some more attention.
While it might be not so critical for us yet, we'd like to support this request very much as we can clearly see how bad this situation is.
I totally upvote this issue!
We are sorry about the slow response times and will try to improve it over time as we scale up the team.
Some automation around testing and publishing could also help if we can manage to get to that.
We feel bad that some plugin updates has had to wait too long. It’s been a struggle finding time to test and review plugins for us but we will take this feedback to heart and see if we can make some changes and share this work among more people on the team.
I've added a project everyone can see to track where plugins are in the publishing pipeline, and will add milestones next with dates.
The intent is to establish a regular publishing timeline.
We'll be adding PRs to a weekly release milestone, and only let them slip when changes are needed before publishing.
Hopefully this will make the whole process cleaner and with as much visibility to plugin authors as possible.
It took around half a year for my plugin to be accepted so 1-month is not a big deal, I suppose :D
I think that this work is not paid by me or anyone (the plugin is not commercial) so I can't expect anything at all. As I understand, Grafana devs were working on Grafana 5 back then, this is why it took so long to accept the PR. New plugin requires a lot of testing. You can help that by providing pre-built images or running instance of your app for testing. This was something that I've done back then. I've built and hosted an app that measures the response time of 500 websites and writes them to Akumuli. That Akumuli instance still available online and can be used for testing.
Things seem to be slipping again, after a time when the updates were being processed really quickly.
Maybe everyone is nursing hangovers from the virtual parties after the conference. :)
We've been developing a tool to validate plugins that will hopefully help speed up the time to review each plugin.
Enter the URL to your plugin release on GitHub and press Enter to validate.
Is there a way to include the community in the PR process somehow? Maybe it would also help to establish a separate community managed plugin catalog or get some advocates to support the Grafana team here?
Here are some of the things we look for during review:
- Try to load the plugin and do some initial smoke testing. Does it work as advertised? Are there any obvious bugs?
- Review the UX. Are there spelling errors? Is it clear to the user how to use the plugin?
- Look at the source code. Are there any tracking scripts or otherwise malicious code?
- Is it a plugin for a commercial product? Commercial plugins need to purchase a subscription.
- Is the plugin trying to track user activity or otherwise spy on the plugin users? Are secrets encrypted or sent in cleartext?
In general, panel plugins are easier to review since you typically don't need to configure any external dependencies. It's also easier to review a new version for a plugin. New data source plugins are typically the most time-consuming, since you often need to configure an instance of the database with the right data.
Community members can help with any of these points. We hope to hire someone to work on this full-time, which should speed up response times.
While it's not directly influencing response times, I've created an updated GitHub project for tracking plugin submissions. Hopefully it will give you a better idea of when you can expect your plugin to be reviewed.
https://github.com/grafana/grafana-plugin-repository/projects/2
@marcusolsson what if the PR is not in the dashboard at all?
Then it means that we haven't triaged the pull request yet. Meaning we need to check whether it passes validation and has the right format. Once that's done, we determine whether it needs more metadata, or is ready to be reviewed.
Could these checks be automated into Github action https://grafana-plugins-web-vgmmyppaka-lz.a.run.app? Or are there other checks that are not supported in that tool yet? I always route my built plugin through it because I know you would refuse to merge it without the tool passing.
They definitely could! The reason we haven't spent more time automating this on GitHub is because we're working on a more long-term solution for submitting plugins, which involves deprecating this repository altogether.
Instead you'll be able to submit your plugin through My Account on grafana.com. In addition to making it a lot easier to submit a plugin, this will let us do more automated UI testing, security checks, and so on, which hopefully will reduce the need for manual inspection (and thus improve response times). The plugin validator will still be a big part of the new process, so please continue using it!
I'll be able to share more information on this in a few weeks! 🤞