Add tests to check all pages load without errors (in different situations)
Opened this issue · 2 comments
We did have some (probably poorly written) tests that I wrote that spidered through the OneZoom site to check all the pages were working (e.g. broken pages caused by stuff like #779). Ideally these would also catch ajax-y type errors like #794 and #799, logging-in errors like #791, and also JS errors like #801.
@lentinj : do you know the status of this sort of testing? Is there any way to start setting up stuff to run on GH PRs that tests at least some of this (it's hard, I guess?). It worries me that we can release a site in which some of this is broken.
As a slight aside, we should also check that hard-coded OTTs, especially the ones in https://github.com/OneZoom/OZtree/blob/main/views/default/footer_sponsor_items.load, are still retained in the database (see #812 for a fix when this failed). We could perhaps do this when building a new tree, e.g. with added instructions in https://github.com/OneZoom/tree-build/blob/main/oz_tree_build/README.markdown#upload-data-to-the-server-and-check-it (point 10).
Chrom(e|ium) headless mode has gotten rid of the faff of webdrivers, and there's various actions for github actions to handle the setup for us, but the fundamental problems that make the tests in tests/functional hard to write and brittle long-term haven't gone away.
We've talked a few times about a minimal tree MySQL dump that's part of the OZtree repo, that could be enough both for new developers to get up to speed without needing persuade a dump out somewhere, and for github actions to be able to configure MySQL and run the tests/unit tests. That would include the controllers/default test, which triggers the controller for each endpoint there.
Once that works we could think about resurrecting the functional tests. If we can make them work locally, github actions shouldn't be too much of a stretch.
Generically loading every page and seeing if there's Javascript errors won't be easy, since a page being "done" doesn't have an obvious answer, beyond something like "wait 3 seconds after the DOM is loaded". But then the tests get very slow, and a github action on every commit is probably the wrong approach.
check that hard-coded OTTs . . . are still retained in the database
This is somewhat similar to the discussion in #742, although having a custom default startpoint was abandoned in the end, configuring these hardcoded OTTs in tree-build-land does feel like the right place for them.