[BUG] ClickablePointOffScreen sometimes fails with RuleExecutionError

[dbjorge]

Describe the bug

An internal customer reported an issue where they were seeing a failure of the ClickablePointOffScreen rule ("An element's IsOffScreen property must be true when its clickable point is off screen.") which was confusing them.

They shared an a11ytest file and the element being reported does have a clickable point which is within the bounds of the desktop, so the rule shouldn't be applying. Even if it was in violation, this rule is warning-level, so it shouldn't be producing an error-level failure.

Spelunking a bit in the a11ytest file shows that the result in question has "Status": 4. This corresponds to the ScanStatus.ScanNotSupported enum value:

ScanNotSupported = 4, // for the cases like HTML framework which we don't support. 

This comment is extremely misleading. The only place where we actually emit this scan status is in response to a RunResult with EvaluationCode = EvaluationCode.RuleExecutionError, which occurs whenever an unknown rule ID is requested to run or when a rule's Condition or PassesTest throws an exception.

In this case, since the Condition should not actually be matched for the element in question (ie, PassesTest should be skipped), my expectation is the Condition is throwing. This rule's Condition is just a call to ClickablePoint.OffScreen. The most suspicious thing I note in that implementation is the use of Desktop.CachedBoundingRectangle. It looks like we're asking UIA to populate the CachedBoundingRectangle value in a Lazy creator callback for the Desktop object only, so I'm suspicious that what may be happening is that the cached values may be getting blown away by a subsequent request for UIA to repopulate the cache with different values (I think DesktopElementHelper.cs might be doing this), but this is just a guess based on code analysis; I haven't confirmed that this is what's happening, and the UIA docs for BuildUpdatedCache are pretty sparse on details (it's not clear whether a call to update the cache for other unrelated properties would destroy the old cached bounding rectangle property or not).

To Reproduce

Repro steps not yet known; see internal email thread for motivating a11ytest file.

Expected behavior

Rule should not apply to the element in question

Actual behavior

Rule's Condition appears to emit an exception

Screenshots or .GIF

See internal email thread

Desktop (please complete the following information):

• OS: unknown
• Accessibility Insights for Windows Version: unknown
• Target Application: see internal email thread
• Target Application Version: see internal email thread

Additional context

Medium priority; results in a false positive but is not otherwise blocking

[msftbot]

This issue has been marked as ready for team triage; we will triage it in our weekly review and update the issue. Thank you for contributing to Accessibility Insights!

[msftbot]

This issue requires additional investigation by the Accessibility Insights team. When the issue is ready to be triaged again, we will update the issue with the investigation result and add "status: ready for triage". Thank you for contributing to Accessibility Insights!

[karanbirsingh]

@RobGallo shared offline that it's unlikely we would pursue a change in the Axe.Windows rule implementation. We have opened microsoft/accessibility-insights-windows#1113 to better address scenarios where rule execution errors are reported in the accessibility error UI.

[microsoft-github-policy-service]

This issue has been marked as being beyond the support scope of Accessibility Insights. It will now be closed automatically for house-keeping purposes.