What are the criteria for inclusion?
david-christiansen opened this issue · 6 comments
@hasufell wrote the following in haskell/error-messages#540:
The readme says:
We are interested in building a catalog of error messages for each tool in common use (GHC, Cabal, Stack, ghcup, HLS, etc)
But why limit to common tools? The idea of easily discoverable error codes goes beyond. Is this a concern of maintenance? Could there be tier3 supported projects that simply get assigned a namespace and can maintain it?
As an example, cabal-cache is currently evaluating whether to follow this scheme: https://github.com/haskell-works/cabal-cache
It's not that common (I guess), but since it's primary use case is CI, it seems very natural in terms of usability to do this. You see an error in your CI logs and can quickly jump to a page explaining it and maybe linking to workarounds.
This is a good discussion to have, so I'm moving it here so we can have it.
Right... I see it as an interesting motivation to encourage the broader ecosystem to adopt this strategy. The best way to encourage may be to give this namespace for free, adding more value to their tools by allowing them to re-use the existing online index and creating an ecosystem standard through network effects.
My concerns are as follows:
- The current UI may get overwhelmed by having too many messages. This is not an issue right now, but it might be someday, and it should be pretty easy to fix.
- I would be afraid that tools that are only used for a short while end up represented in the index for all time. I think this is also solvable by UI.
- The namespace might be a somewhat scarce resource to keep track of. This is a minor concern.
Basically, as long as something is a tool used by more than one Haskell developer, I'm generally in favor of including it here. The drawbacks to lots of support are minor, and mostly something that some programming can solve.
And we can always delete things if they become completely irrelevant and are somehow a burden - if we had 100 random research projects from grad students from 2002 that can't be found on the Internet at all, then it might be worth considering a cleanup. But even then, I'd probably just want to suppress them in certain views (like have a "show rarely-used projects" checkbox or something)
And I've never asked for money for a namespace slot - I just want to administer them in that text file so that we avoid conflicts, e.g. if Stan and Stack both wanted S-.
There could be some barriers towards inclusion:
- project must exist on hackage
- must be actively maintained (no one-off or toy projects)
- some minor proof of popularity (e.g. 20+ stars on github, download count on hackage, ...)
- executables only (for now, was this discussed at all?)
Since this would be an explicit request based system, I doubt there will be overwhelming interest at first.
I don't even think that formal criteria are needed - I think a judgment call is fine. For not-yet-popular tools, I think the best criterion would be "we'll add you if you document N messages", where N is something like 5.
So far, adding new tools has had a bit of technical cost from my side in generalizing the site generator. I'm happy to bear this cost for important core tools like GHC, GHCup, and Stack, but I'd rather have someone else do so for less-popular projects. As the site matures, this will also get easier and easier. Supporting GHCup was very easy (maybe 30 min work), while adding Stack support took me about 2 hours.
Agreed about the network effects and the value of a common place for all the tools. My thought process has always been that if an error occurs in a build, users shouldn't have to try to puzzle out where it came from. A single source of explanations and workarounds is just much easier to use than having one for each tool. This is where I hope we'll be ahead of the competition in other languages :-)