cypress-io/cypress

When downloading 2 versions of Cypress simultaneously, one clobbers the other resulting in nondescript error and exit

jennifer-shehane opened this issue · 12 comments

jennifer-shehane commented

Current behavior:

We recently encountered an error on one of our repos ourselves where we use yarn. The problem was that we were downloading two versions of cypress within our package.json. We run a monorepo, so one of the cypress versions was in the main package.json and the other cypress version was in a sub-module of our monorepo.

Since we save cypress upon download to /tmp/cypress.zip, they were clobbering each other when both were being downloaded/unzipping at the same time. Due to how yarn works, this is likely to exhibit more frequently in it - although it's still possible using npm.

Note: This can still happen even if the 2 Cypress versions are the same version number

Basically:

  • Version 1 and 2 of Cypress begin download
  • Version 1 of Cypress finishes download
  • Version 2 of Cypress is still downloading - deleting current /tmp/cypress.zip
  • Version 1 of Cypress begins unzipping /tmp/cypress.zip
  • Error

Resulting in this error specifically:

Installing Cypress (version: 3.3.2)

[04:46:10]  Downloading Cypress     [started]
[04:46:11]  Downloading Cypress     [completed]
[04:46:11]  Unzipping Cypress       [started]
read err Error: not enough bytes in the stream. expected 8709456. got only 2882632
    at AssertByteCountStream._flush (/root/monorepo/node_modules/yauzl/index.js:494:15)
    at AssertByteCountStream.prefinish (_stream_transform.js:141:10)
    at AssertByteCountStream.emit (events.js:198:13)
    at prefinish (_stream_writable.js:635:14)
    at finishMaybe (_stream_writable.js:643:5)
    at endWritable (_stream_writable.js:663:3)
    at AssertByteCountStream.Writable.end (_stream_writable.js:594:5)
    at InflateRaw.onend (_stream_readable.js:655:10)
    at Object.onceWrapper (events.js:286:20)
    at InflateRaw.emit (events.js:203:15)
    at endReadableNT (_stream_readable.js:1129:12)
    at process._tickCallback (internal/process/next_tick.js:63:19)
events.js:174
      throw er; // Unhandled 'error' event
      ^

Error: invalid central directory file header signature: 0x98d05ebb
    at /root/monorepo/node_modules/cypress/node_modules/yauzl/index.js:258:70
    at /root/monorepo/node_modules/cypress/node_modules/yauzl/index.js:631:5
    at /root/monorepo/node_modules/cypress/node_modules/fd-slicer/index.js:32:7
    at FSReqWrap.wrapper [as oncomplete] (fs.js:467:17)
Emitted 'error' event at:
    at emitError (/root/monorepo/node_modules/cypress/node_modules/yauzl/index.js:232:8)
    at emitErrorAndAutoClose (/root/monorepo/node_modules/cypress/node_modules/yauzl/index.js:227:3)
    at /root/monorepo/node_modules/cypress/node_modules/yauzl/index.js:258:42
    at /root/monorepo/node_modules/cypress/node_modules/yauzl/index.js:631:5
    at /root/monorepo/node_modules/cypress/node_modules/fd-slicer/index.js:32:7
    at FSReqWrap.wrapper [as oncomplete] (fs.js:467:17)
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
Exited with code 1

See #3515 (comment)

Desired behavior:

When downloading Cypress, we need to make the tmp destination more unique like:

/tmp/cypress-${version}-${random}.zip

Steps to reproduce: (app code and test code)

project/packages/foo/package.json

{
  "name": "foo",
  "devDependencies": {
    "cypress": "3.3.2"
  }
}

project/packages/bar/package.json

{
  "name": "bar",
  "devDependencies": {
    "cypress": "3.3.2"
  }
}

Run yarn

Versions

Cypress 3.3.2

xeger commented

We're running into this too! I hypothesized that something was being clobbered, but hadn't gotten very far into it yet.

I happen to be using Cypress inside a container, so my current workaround is to make my base image cypress/included:x.y.z and then to force npm/yarn not to reinstall the damned thing (and ideally, not to install any other versions concurrently!)

wmartins commented

Ran into the same issue when setting up a monorepo strategy in one of my company's project.

In fact, in my case, the problem doesn't occur as specified by @jennifer-shehane. In my case, it only occurs because I'm installing two different versions of Cypress (3.3.2 and 3.4.0 in my tests here).

So, I think some things can help out yarn monorepo users:

  • Try to "align" your Cypress versions in all packages in your monorepo (which may be challenging)
  • If you want a quick fix, you can set CHILD_CONCURRENCY to 1. This will download Cypress sequentially, which may slow down your build, but at least it will not fail when installing Cypress

In my case, I changed my testing strategy to centralize e2e tests into a single place instead of having it divided by package. It was a possibility in my case.

  • 👍7
stoikerty commented

Our solution to this issue (which might not apply to everyone) is to install cypress alongside lerna in the root of the monorepo. Hope this solution helps someone in need 😄

Since cypress is installed together with lerna and lerna is only able to run its tasks after that install, there are never conflicts with subsequent cypress installations since the binary already exists when the install on individual packages are run with lerna.

We use CircleCI, here's a modified extract of our config.yml. Note the use of working_directory and CYPRESS_CACHE_FOLDER and how the defaults are used together with persisting and attaching the workspace.

version: 2

defaults: &defaults
  working_directory: ~/working_directory
  environment:
    CYPRESS_CACHE_FOLDER: '~/working_directory/.cache/Cypress'
  docker:
    - image: circleci/node:12.14.0-browsers

jobs:
  install:
    <<: *defaults
    timeout: 1000
    steps:
      - run:
          name: Install Dependencies without updating package-lock.json
          command: npm install --no-save
      - run:
          name: Install package dependencies
          command: npm run bootstrap -- --ci
      - persist_to_workspace:
          root: ~/working_directory
          paths: ./**

  test:
    <<: *defaults
    steps:
      - attach_workspace:
          at: ~/working_directory
      - run:
          name: Running tests
          command: npm test
...
  • 👍1
krokofant commented

I worked around this by setting env var CYPRESS_INSTALL_BINARY to 0 to skip downloading the cypress binary while installing packages, and then running yarn cypress install to install the cypress binary.

  • 👍6
Vages commented

I could have a go at solving this. Is that OK, @jennifer-shehane?

pmgmendes commented

Following @wmartins first suggest approach I've used selective version resolutions by adding cypress entry in the "resolutions" array. All the direct or transitive dependencies will resolve to the top level cypress version, in my case, cypress@4.10.0.

  • 👍3
Vages commented

For all my suggestions on circumventing the problem, see "Fixing/circumventing the problem" at #7951

jennifer-shehane commented

@Vages Yes, feel free to an open a PR - even one that is a 'draft' is fine. Check out our contributing doc before getting started, but mostly we're pretty big on adding tests around what the PR is fixing 😉.

tomevans18 commented

There seems to be a lot of comments in relation to mono repos but I don't think this issue is limit to this setup.

Within our repo we have the following dependancies:

  • cypress (5.6.0)
  • @testing-library/cypress (7.0.1)
  • @types/testing-library__cypress (5.0.8)

When inspecting the yarn lock you will notice that @types/testing-library__cypress has a dependancy of cypress "*"

"@types/testing-library__cypress@5.0.8":
  version "5.0.8"
  resolved "https://registry.yarnpkg.com/@types/testing-library__cypress/-/testing-library__cypress-5.0.8.tgz#6a8effce8218c5be42c1775b26dcbdcc273fda62"
  integrity sha512-1dTaoaCR6PWVRKXZAHcWvnAzHpTDKUKWw1pq4yzyoyI/RQ4KRySA9EFLwS0uEugTIPPgBaPUnXK5OwTaQoQp7g==
  dependencies:
    "@testing-library/dom" "^7.11.0"
    cypress "*"

This results in:

cypress@*:
  version "6.0.0"
  resolved "https://registry.yarnpkg.com/cypress/-/cypress-6.0.0.tgz#57050773c61e8fe1e5c9871cc034c616fcacded9"
  integrity sha512-A/w9S15xGxX5UVeAQZacKBqaA0Uqlae9e5WMrehehAdFiLOZj08IgSVZOV8YqA9OH9Z0iBOnmsEkK3NNj43VrA==

In addition to:

cypress@5.6.0:
  version "5.6.0"
  resolved "https://registry.yarnpkg.com/cypress/-/cypress-5.6.0.tgz#6781755c3ddfd644ce3179fcd7389176c0c82280"
  integrity sha512-cs5vG3E2JLldAc16+5yQxaVRLLqMVya5RlrfPWkC72S5xrlHFdw7ovxPb61s4wYweROKTyH01WQc2PFzwwVvyQ==

This obviously results in two versions of Cypress being downloaded, resulting in this issue you have described, but in reality the 5.6.0 version should be the only one downloaded/used for the wildcard dependancy as it meets the requirements.

For this reason I don't truly believe this issue is solely with Cypress but possibly with Yarn and/or other libs that use * version dependencies.

Yarn seems to be aware of this issues - yarnpkg/yarn#6695

  • 👍2
jennifer-shehane commented

Providing a more complete example of this to test if the PR opened will fix it:

  • Run cypress cache clear to clear any downloaded Cypress caches previously and delete any existing yarn.lock file
  • yarn i with project below.

project/package.json

{
  "name": "ex-project",
  "private": true,
  "version": "1.0.0",
  "main": "index.js",
  "license": "MIT",
  "workspaces": ["packages/*"],
  "scripts": {}
}

project/packages/foo/package.json

{
  "name": "foo",
  "private": true,
  "version": "1.0.0",
  "devDependencies": {
    "cypress": "6.3.0"
  }
}

project/packages/foo/package.json

{
  "name": "bar",
  "private": true,
  "version": "1.0.0",
  "devDependencies": {
    "cypress": "6.4.0"
  }
}
cypress-bot commented

The code for this is done in cypress-io/cypress#14952, but has yet to be released.
We'll update this issue and reference the changelog when it's released.

  • 🎉1
cypress-bot commented

Released in 6.5.0.

This comment thread has been locked. If you are still experiencing this issue after upgrading to
Cypress v6.5.0, please open a new issue.