Nyc test working properly locally but failing in CI / Docker environment

[8byr0]

Hi there!

I'm having bad times generating code coverage in my CI. There's an error which I'm unable to troubleshoot there and it only occurs when wrapping my test command in NYC:

error An unexpected error occurred: "Cannot read property 'indexOf' of undefined".

Let's summarise my config:

Package.json

// package.json - relevant parts
{
  "scripts": {
    // Other scripts
    "test": "cucumber-js",
    "test:coverage": "nyc --reporter=lcov yarn test"
  },

  "dependencies": {
    "@babel/plugin-transform-runtime": "^7.17.0"
  },
  "devDependencies": {
    // Babel
    "@babel/core": "^7.17.5",
    "@babel/plugin-transform-modules-commonjs": "^7.16.8",
    "@babel/preset-env": "^7.16.11",
    "@babel/preset-typescript": "^7.16.7",
    "@babel/register": "^7.17.0",

    // Cucumber
    "@cucumber/cucumber": "^7.3.2",
    "@cucumber/pretty-formatter": "^1.0.0-alpha.1",
    
    // Istanbul
    "@istanbuljs/nyc-config-typescript": "^1.0.2",
    "nyc": "^15.1.0",

    // Reac testing library
    "@testing-library/dom": "^8.11.3",
    "@testing-library/jest-dom": "^5.16.2",
    "@testing-library/react": "^12.1.3",

    // Chai
    "@types/chai": "^4.3.0",
    "@types/chai-dom": "^0.0.12",
    "chai": "^4.3.6",
    "chai-dom": "^1.11.0",

    // Typescript
    "ts-node": "^10.6.0",
    "tsconfig-paths": "^3.13.0",
    "typescript": "^4.6.2",
    "@types/node": "^17.0.21",

    // Utils
    "dotenv": "^16.0.0",
    "isomorphic-fetch": "^3.0.0"
  },
  "peerDependencies": {
    "babel-loader": "^8.2.3"
  }
}

cucumber.js (default file used by cucumber)

// cucumber.js

const defaults = [
    '--require cucumber.setup.js',
    // Global hooks
    '--require cucumber/runner.js',
  
    // Test files
    '--require __tests__/**/**/*.spec.ts',
    '--require __tests__/**/**/*.spec.tsx',
    '--require src/**/**__tests__/**/**/*.spec.ts',
    '--require src/**/**__tests__/**/**/*.spec.tsx',
    // Features
    '**/**__tests__/**/**features/**/*.feature',
    '--publish-quiet',
  ].join(' ');
  
  module.exports = {
    ci: `
    --world-parameters '{"ci": "true"}'
    ${defaults}
    --exit
    `,
    default: `
    ${defaults}
    `,
  };

// cucumber.setup.js
/* eslint-disable @typescript-eslint/no-var-requires */
/* eslint-disable import/no-extraneous-dependencies */

require("isomorphic-fetch");

const ignoredModules = ["next/server"].join("|");

require("@babel/register")({
  plugins: ["relay"],
  presets: ["next/babel", "@babel/preset-typescript"],

  extensions: [".js", ".ts", ".tsx"],
  exclude: [`node_modules/(?!${ignoredModules})`],
});

require("ts-node").register({
  transpileOnly: true,
  ignore: [`node_modules/(?!${ignoredModules})`],
});

require("tsconfig-paths").register();

require("jsdom-global/register");

.nycrc.json

// .nycrc.json
{
  "extends": "@istanbuljs/nyc-config-typescript",
  "all": true,
  "check-coverage": true,
  "lines": 10,
  "require": [
    "./cucumber.setup.js", // Using the same config as cucumber 
    // Also tried with a dedicated one using only ts-node/register
    // see "nyc.setup.js" below
  ],
  "report-dir": "./coverage",
  "source-map": true,
  "instrument": true,
  "produce-source-map": false,
  "include": [
    "src/**"
  ],
  "excludeNodeModules": false
}

// nyc.setup.js - not in used but tried too
const ignoredModules = ["next/server"].join("|");

require("ts-node").register({
  transpileOnly: true,
  ignore: [`node_modules/(?!${ignoredModules})`],
});

require("tsconfig-paths").register();

Here's the problem:

Locally (OSX terminal)

✅ yarn run test # cucumber-js
✅ yarn run test:coverage # nyc --reporter=lcov yarn run test

Locally (Docker container)

- ✅ yarn run test # cucumber-js
- ❌ yarn run test:coverage # nyc --reporter=lcov yarn run test   (see error below)

Result of test coverage command in docker container:

root@f90d3f14030e:/app# yarn run test:coverage                                                                                                                                                                                            
yarn run v1.22.15
$ nyc --reporter=lcov yarn test
$ cucumber-js
error An unexpected error occurred: "Cannot read property 'indexOf' of undefined".
info If you think this is a bug, please open a bug report with the information provided in "/app/yarn-error.log".
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
ERROR: Coverage for lines (0%) does not meet global threshold (10%)
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

I realised this issue after a my gitlab CI pipeline failed and made a reproduction easily using docker locally.

I do not really suspect NYC but I have absolutely no clues how to debug this so:

• Is there a way do get verbose output from what NYC is doing?
• Is it possible that with my config above, I end up running NYC nested? (Which would be surprising cause it works properly locally)
• Any other idea?

A few more:

• I suspected a failure due to headless environment (cause I use jsdom to test a NextJS app) but since the standard test command (i.e. yarn run test) does not fail in CI I would not blame that
• I tried a fresh clone of my git repository to ensure no cache would remain from previous runs and it's the same (it works properly but not in docker env)

Thanks for your hints !

[eulersson]

I'm experiencing exactly this at the moment you merge coverage maps cypress-io/code-coverage#455

Here's the calls the hang my CI provider: https://github.com/cypress-io/code-coverage/blob/3c280862861d00addb38f02e7b488e3aa24b75ca/task.js#L145-L168