User selected "Is Higher/Better" is too subjective.
Closed this issue · 10 comments
Having user defined/selected "is better" field seems... dangerous. I think as per the goals of the platform is to present the benchmarks with as little interpretation/editorialization as possible. It would be better IMHO to leave this field out and provide the user ways to sort the submissions (as is already done) and leave the interpretation up to them.
Ah, I think I see what you mean here. Although I think in this context anyway, the "is higher better" field is intended to be interpreted as whether we should deem a positive score (let's say 100) as better than let's say a negative score (something like -100). Perhaps for some metrics, the lower the value is, the better the metric should be interpreted to be performing (and vice versa for the other direction).
However, I think your comment is still relevant for a couple of other reasons as well. For one, the terminology "is higher better" could definitely be interpreted to be oddly phrased or ambiguous perhaps. PWC does seem to have a similar terminology in their modal, but that doesn't necessarily mean we can't improve upon it:
It's valuable input that you provided in any case as this was your impression of what the checkbox meant, this is perhaps something that we need to clarify. I think keeping the checkbox in there makes sense, but perhaps a more clear field name? Or maybe some helpful hovertext? Any preferences/thoughts on that?
It's as Vincent says: we're borrowing PWC's approach to defining the directionality of a user-supplied metric, not asking the user to say that this is the best metric value result. However, it's not very clear in PWC, and it's not very clear here. Perhaps we can add a better explanation, like as a "tool-tip."
Yeah a tooltip would be helpful. Alternately, direct labeling with radio buttons and something like
Better performance for this metric is indicated by:
() higher values.
() lower values.
Side question: Is it true that all metrics will have a numeric value? Could there be bools, enums, etc? If so, this choice would be conditional on numeric values.
Yeah, I think a tooltip or more explicit and separate labelling would be helpful here. As Dan mentioned, I was also confused by this as well, so this seems to be a common point of confusion.
As for the side question, that's a good one. I honestly cannot definitely say for sure on that. I would imagine that this could be pretty flexible, so perhaps an integer is not sufficient to cover all possible cases here. I suppose there's a bit of an art in balancing the line between specificity that is a bit too rigid and freedom that is a bit too unconstrained and ambiguous. As I don't claim to be an expert in the realm of quantum benchmark metrics, I'm not sure I can definitively say one way or another, but I think your side question still stands and is relevant here.
Since I figured out how to include Bootstrap tooltips, yesterday, I'll add one, now. In fact, we might as well add them to every modal input field.
As for the side question, firstly, PWC only allows numerical metric values. Secondly, this is semantic, but a literal "metric" is a measure of distance that obeys the triangle inequality, and that is what we mean, here, I think. There is such a thing as a "discrete metric," where binary true/false values would be appropriate, but I have to defer to @willzeng and the rest of the team on whether there will be any demand for this case, or others.
I like the suggestion from @crazy4pi314 of adding more clarification to
Better performance for this metric is indicated by:
() higher values.
() lower values.
I don't have a strong use case for anything other than numeric values.
I started adding tooltips to everything, which I'm finding doesn't clutter the interface, while it lets us add verbose explanations to all form field labels. With the exception of login and registration, if we go this route, we should probably set the standard that all form field labels carry tooltips, so that the user knows they can always count on these verbose explanations being there, on mouse hover. I think, as we have them right now, it's not likely that a user could fail to notice their presence, for very long at all.
I liked the 2-option radio button group suggestion, as well, and I was about to implement it after tooltips. However, now that the tooltips are in, I wonder if this is still necessary. (Please take a look at QA, to assess what I mean.)
Usually, a 2-option radio button group is considered an anti-pattern, exactly because this is why a checkbox exists. It's still the better option, if it more clearly communicates the boolean value's meaning to the user. However, with the tooltips, I think it might be overkill.
Please take a look.
I just added a bold but unobtrusive "(Mouse-over labels for explanation.)" line at the bottom of the modal, in case the user would otherwise miss the tooltips. I think they also work fairly well on mobile.
I love these, great work! And good point on the anti-pattern 👍
Then, I think we can close this issue, for now, since it seems like we've made it clear to the user what we're asking for with this checkbox and why. Thank you, everyone.