Change the way "Run All Specs" are run in Interactive Mode (from the GUI)
brian-mann opened this issue · 8 comments
This comes from this epic... #681
All other stories were complete except this section...
Interactive Mode
- Continue enabling users to "Run all Tests"
- Enable clicking folders to automatically run a subset of specs
- Instead of killing the renderer process between specs the browser will simply be "navigated" to the next spec URL.
- This will isolate each spec, and be much faster and less abrupt than killing the renderer like we will for the CLI. However this makes sense considering Cypress already has the code for swapping the URL and rehydrating the state of the reporter. We already do this when swapping domains in a cy.visit.
- Isolating specs will help with memory exhaustion, but its still possible to blow out on a very long and intense spec file.
- To solve this we'll begin purging snapshots by reducing the default minimum of 50, down to 5.
- Purged snapshots will only affect passing tests and we will not purge data for failing tests. This will enable users to review the commands and output for failed tests - as they likely don't care about the data for passing tests anyway.
- We'll need to add a good user experience / messaging around why snapshots were purged
- When running multiple specs we'll add the spec file as part of the command log which will give you a shortcut for focusing in on a spec file.
@jennifer-shehane we'll need you to comp some UI changes related to communicating to the users about running too many tests in Interactive Mode.
This should include a banner saying: "It looks like you're trying to run all tests in Interactive Mode. To prevent crashes slowdowns we are turning off interactive snapshots and debugging features."
I have a question about this point:
- To solve this we'll begin purging snapshots by reducing the default minimum of 50, down to 5.
Is there an API/config setting to do this in the current version?
Yes its numTestsKeptInMemory
Comp for:
- Explaining that/why snapshots are disabled (with link to docs).
- List spec file within command log
@brian-mann mentioned maybe adding a tooltip message when you hover over the commands, since you are likely trying to snapshot if you're hovering.
Added a tooltip in the style we have tooltips today.
Do I understand correctly that you will soon alter the behaviour of cypress open and cypress run in such a way that cypress open will behave like cypress run?
After that, will we lose the ability to run before hook really once before all tests?
What is the status of this issue? This is a blocker for the cypress-image-snapshot library we would like to implement in our codebase
This seems like a really good improvement. Especially the running a subset of specs by folder and not killing the browser unnecessarily in open mode. So here is a +1 from me.
Turning off test isolation fix issue in my test setup.