facebook/react

[RFC] Testing Browser Compatibility

nhunzaker opened this issue · 19 comments

nhunzaker commented

Testing browser quirks is challenging. What if we set up a page or site local to the React repository that made it easier to isolate browser differences and document known issues:

For example:
http://natehunzaker.com/react-issue-reproducer/cases/input-kitchen-sink/index.html

We could start with some of the outstanding input issues (like #7359), and work our way backward to historical decision points for adding additional rules to support specific browsers.

Speaking with @gaearon and @aweary about this. We see a couple of outstanding tasks:

  • Add a page/site local to the React repository that includes test cases for reproducing specific browser quirks
  • Make it easy to switch between versions of React via CDN references and local builds
  • Catalog existing fixes and outstanding issues, start building test cases
  • Modify React flags via query strings, or some other mechanism to quickly switch between the Fiber and and standard DOM renderer.

Eventually, this could also be a great resource for how React handles DOM manipulation, obscure DOM API concepts, and reference notes for implementors of other DOM rendering libraries.

Test cases

  • #7328: IE Password fields - @nhunzaker
  • #8575: Backport input fix - @jquense
  • #4672: Change event does not fire in IE and Edge under certain circumstances - @jquense
  • #7027: OnChange randomly misses keys on IE if typing very quickly - @jquense
  • #8308: onClick firing for disabled buttons - @aweary
  • #4457: IE11 block change events if disabled @aweary
  • #5762: Mouse enter and leave work consistently with disabled elements @aweary
  • #708 Change event bubbling doesn't work with controlled components in IE
nhunzaker commented

cc @jquense, you've done a lot of work in this area too. What do you think?

aweary commented

A few thoughts/questions on the high-level structure of the page(s):

  • How should we group test cases? Should each case have its own page and we maintain a central directory for navigating between them, or should we keep the page count down and put all related tests on a single page?

  • We'll want to include exact steps on what to do to reproduce and check the behavior we're looking to validate. It should be easy for a new contributor to look at it, follow the steps, and know if its working or not.

  • We can potentially use paths to load any arbitrary version of React from unpkg on any page. For example /15.4.1/input-test would load the test with React 15.4.1. This would make it very easy + quick to test regressions.

gaearon commented

I would put everything on a single page and have a switch at the top that points to CDN versions or local build, and refreshes the part of the page below when you change them.

bvaughn commented

Make it easy to switch between versions of React via CDN references and local builds

I have a small set of in-browser tests for react-virtualized that read query param and toggle between different versions of react (by version number) and react-virtualized (either a version number pulled from CDN or "local" which runs against the local build). Very useful when checking for regressions or perf changes.

+1 to Dan's in-page switch suggestion. The most common case is probably to compare between local and latest release on NPM.

jquense commented

this sounds really helpful in general. So much of that knowledge now is either lost, or trapped in tons of jsfiddles in many different issues. I'd love if the many many browser decisions made to attend to browser quirks was better documented, ITs always tough to try and understand even the starting point for addressing DOM issues without that knowledge. I'd also add that documenting the minimal reqs for certain changes would be really helpful. Such as input components needing to work with imperative sets to value on the DOM node, as well as prop changes.

Would this also be a tool for new issues? I wonder what adding to such a page would look like? Changes accompany associated fixes? Is it possible to leverage an existing code iteration tool (codepen/jsbin/jsfiddle) for this? That might make it more helpful for the beginning bits of identifying new issues.

  • 👍1
aweary commented

As I see it, the process would look something like:

  1. User identifies new issue and shares a case reproducing it (JSFiddle or whatever)

  2. We verify the issue

  3. Someone fixes it and includes the changes to the test page in the PR just like they do now with unit tests

  4. We now use the test page to validate the fix in our supported browsers

aweary commented

This is a long term stretch goal, but automatic deployment of the test page when a PR is opened that affects DOM code would be amazing.

nhunzaker commented

One other item, though I'd be okay with excluding it as a core part of this issue, is base line browser compatibility.

  • What browsers do we target?
  • Should there be an effort to strip fixes for unsupported browsers, such as IE8?
  • If not, should we consider a compatibility package? I think this primarily pertains to the event plugin system, though I could be wrong.
aweary commented

#8589 has been merged! The DOM fixtures application is in place and functioning, now we need to focus on writing up all the fixture test cases and designing the UI for them

jquense commented

Some outstanding DOM issues, i'm personally familiar with are ('ill try and collect others as well)

  1. #5762 (I had an offline chat with @jimfb awhile ago about this and I believe it was left at "looks fine but we are all to scared to make such a big change to the mouse enter logic", so probably a good fit here.

  2. #8575 Really is a collection of undocumented input behavior that needs documentation.

  3. #4672

  • 👍2
aweary commented

A few issues that will make good fixtures: #7027, #4457, #8308

jquense commented

We still need to write the fixture components yeah? I could take a stab at that if its not in process somewhere

aweary commented

@jquense yupp, go for it! ping me when you've got a PR open and I'll review

nhunzaker commented

Started to catalog a check list of test cases in the issue description.

  • 👍2
nhunzaker commented

I'm a bit stuck on this one:

Modify React flags via query strings, or some other mechanism to quickly switch between the Fiber and and standard DOM renderer.

Does React expose the ability to touch feature flags with the UMD build?

nhunzaker commented

@jquense @aweary I added your names next to the checklist of test cases to write based on the work progress. If I just created work for you, feel free to remove your names :).

aweary commented

@nhunzaker sounds good, I claimed the unclaimed ones 😃

gaearon commented

Can we close this now? 😃

  • 👍1
nhunzaker commented

Yes. I think we certainly can.