Cypress tests have become flaky
Closed this issue · 2 comments
It appears that several cypress tests have become quite flaky, in particular:
-
The initial smoke test sometimes fails with a connection timeout, presumably because the webpack-dev-server hasn't finished starting up yet. I tried adding a
wait-on http://localhost:8082pretest call, but it hasn't seemed to help. -
The last test in the Thematic Layer suite sometimes fails with a "cannot read property 0 of undefined" error. This is actually indicative of a real bug. It happens because the multiple "dimension" querystring parameters added to anEDIT I believe I've fixed this now, it was a different issueanalyticsAPI request URL could show up in any order - since we only have one order available in the Cypress Network Fixture, that's the only one that succeeds (the others hit a 404 error and the application breaks, which it probably also shouldn't do)
These are quite obnoxious because they prevent, most importantly, the application from being deployed to core from master. I was able to do this manually by restarting the build on Travis multiple times, but that's not a practical long-term strategy.
I believe I've fixed issue #2 - it appears to have actually been a race condition between the test run finishing and the error occuring. Solved by manually adding the relevant end-of-test fixtures which weren't captured by the yarn cy:e2e:update process.
This has been fixed using wait-on http-get://localhost:8080 to wait for GET instead of HEAD