currents-dev/cypress-cloud

With parallel enabled, cypress-cloud sometimes don't reporting results and artifacts to sorry-cypress

Opened this issue · 0 comments

Please confirm

  • I have searched for duplicate or closed issues and discussions.
  • I will include a minimal, self-contained set of instructions for consistently reproducing the issue.
  • I will attach a full debug log, otherwise the issue will be closed with no response.

Environment information

# Put output below this line
  System:
    OS: Linux 5.14 Debian GNU/Linux 11 (bullseye) 11 (bullseye)
    CPU: (16) x64 Intel Xeon Processor (Cascadelake)
    Memory: 13.25 GB / 31.09 GB
    Container: Yes
    Shell: 5.1.4 - /bin/bash
  Binaries:
    Node: 20.5.0 - /usr/local/bin/node
    Yarn: 1.22.19 - /usr/local/bin/yarn
    npm: 9.8.0 - /usr/local/bin/npm
  Browsers:
    Chrome: 114.0.5735.133
  npmGlobalPackages:
    corepack: 0.19.0
    cypress: 12.17.4
    npm: 9.8.0
    typescript: 5.2.2

Describe the bug

Trying to switch from Cypress docker image to Currents docker image.

Related packages:

  1. sorry-cypress: 2.6.0
  2. currents docker image: currentsdev-cypress-included:12.17.4
  3. cypress-cloud: 1.10.2
  4. cypress: 12.17.4

With parallel enabled, cypress-cloud sometimes don't reporting results and artifacts to sorry-cypress, this results to test runner hanging and finally sorry-cypress timed out with the specified run.

Strangely, if I enabled "@cypress/code-coverage", the whole process works fine. The hanging issue happens when I don't use "@cypress/code-coverage" with the same configuration and execution environment.

A little background: before this switch, I use "cypress-included:12.7.0" docker image with the same sorry-cypress service without "@cypress/code-coverage" plugin. It works fine for a long time.

Expected behavior

With parallel enabled, cypress-cloud always reporting results and artifacts to sorry-cypress if no error happens.

Setup and Command

// Core config with some information (not related or sensitive) removed.
const config = defineConfig({
  e2e: {
    testIsolation: false,
    defaultCommandTimeout: 15 * 1000,
    pageLoadTimeout: 2 * 60 * 1000,
    requestTimeout: 15 * 1000,
    experimentalStudio: true,
    specPattern: specPattern,
    excludeSpecPattern: excludeSpecPattern,
    async setupNodeEvents(on, config) {
      require('@cypress/code-coverage/task')(on, config);
      const finalConfig = await cloudPlugin(on, config);
      return finalConfig;
    },
  },
  retries: {
    runMode: 2,
    openMode: 0,
  },
});

// Plugins/packages used:
//   "cypress-cloud"
//   "cypress-visual-regression"
//   "@cypress/code-coverage"
//   "cypress-wait-until"

// Command to run the tests (4 parallel runs):
//   $ npx cypress-cloud --parallel --record --key ${the_key} --ci-build-id ${the_ci_build_id} --browser chrome

Full log and debug output

// Put your logs below this line

I am sorry, since it runs without error and the output is too long, I don't include the full log.
If you need more information about this, please let me know, thanks!

By observing the log, I found that the normal tests output a log like this:

    "Reporting results and artifacts in background..."

But the "lost" tests don't output such log.