cypress-io/cypress

Inconsistent beforeEach behavior when 'Run all specs' executed.

lgandecki opened this issue · 9 comments

lgandecki commented

Current behavior:

As one of cycpress-cucumber-preprocessor plugin users found out:
badeball/cypress-cucumber-preprocessor#139

the beforeEach defined in one file runs for every test.

Having one file:

beforeEach(() => {
  cy.log("This should run only before the first one");
});

describe("abcd", () => {
  it("dfsdfs", () => {});
});

and another:

beforeEach(() => {
  cy.log("This should run only before the second one");
});

describe("abc", () => {
  it("dfsdf", () => {});
});

I would expect them to stick to their unique beforeEaches, and that's how it works when running them separately, and when running them together using cypress run. With cypress open and the "run all specs" function - they both run both beforeEaches.

Desired behavior:

Consistency between runing specs separately and all together.

Steps to reproduce: (app code and test code)

reproduction here: https://github.com/TheBrainFamily/cypress-cucumber-example/tree/beforeEach

Versions

as in the repro - I used macos

jennifer-shehane commented

Related to: #1586

xeger commented

My team encountered this while working with vanilla Cypress (not the Cucumber plugin).

Even more surprising, the behavior of Cypress varies depending on how it is invoked!

  • cypress open behaves as this issue describes; there is no isolation between test files
  • cypress run seems to reset the Mocha context between test files; each file only runs its own hooks

Based on the type:bug status, it sounds like the intention of Cypress is to isolate different test files?

  • 👍6
bahmutov commented
xeger commented

The "leaky" behavior (no isolation) only happens when I click "run all tests." If I click an individual test file and later click another, isolation is maintained.

Seems to me that "run all tests" should behave more like cypress run if possible.

  • 👍7
technomage commented

If the non-cucumber case encloses them in describe blocks do you see this behavior? In mocha use describe blocks for scope. I will test out locally.

technomage commented

Yep. If the beforeEach are within a describe block they are isolated. The cucumber template should place a describe block around the entire file content to isolate.

  • 👍1
napervomay commented

Still reproducible on versions 7.2.0, 7.4.0 and 8.4.1.
No Cucumber, just Cypress with Mocha.

Same steps to reproduce:

  • open GUI
  • click 'Run * integration specs'
emilyrohrbough commented

Run All specs functionality was removed in Cypress 10.0.0. See #21628