karma-runner/karma-coverage

Karma exits with code 0 when the coverage threshold is below the configuration

jesusbarcenilla opened this issue · 8 comments

jesusbarcenilla commented

Hi,

I added this configuration to my karma.conf.js

            type: 'html',
            dir: 'test/coverage/',
            check: {
                global: {
                    statements: 85,
                    branches: 70,
                    functions: 85,
                    lines: 85
                }
            }

The logs work fine and if the thresholds are not met, I can see error logs in the terminal but karma always exit wit code 0. That affect my CI process that is not able to detect something is wrong.

The documentation says

If the thresholds are not met, karma will return failure.

and I've review the code and I've seen the plugin is setting the exit code to 1 so I'm not sure if it is a issue in karma-coverage or in karma itself.

I'm using :
karma@5.1.1
karma-coverage@2.0.3

Epoxide commented

I'm experiencing the same.

karma.conf.js

reporters: ['coverage', 'spec'],

 coverageReporter: {
    includeAllSources: true,
    dir: 'coverage/',
    reporters: [
         { type: "html", subdir: "html" },
         { type: 'text' },
         { type: 'text-summary' }
     ],
     check: {
          global: {
              statements: 80,
              lines: 80,
              branches: 80,
              functions: 80
          }
     }
 },

Karma output:

07 08 2020 13:41:20.282:ERROR [coverage]: Chrome Headless 85.0.4182.0 (Linux x86_64): Coverage for branches (66.96%) does not meet global threshold (80%)
=============================== Coverage summary ===============================
Statements   : 81.5% ( 1674/2054 )
Branches     : 66.96% ( 612/914 )
Functions    : 85.93% ( 281/327 )
Lines        : 81.63% ( 1653/2025 )

Karma exits cleanly with 0. I've tried emitWarning: false inside check with no luck (even though this is not documented in karma-coverage). I'm running:

karma: 5.1.1
karma-coverage: 2.0.3
matz3 commented

See #418
Issue is still not solved

jesusbarcenilla commented

@matz3 It looks like your PR was already merged in master. I've tested it and it works fine.
Looking forward for the next karma release.
I'll keep this issue open until then.

Thank you very much.

  • 👍1
industral commented

still doesn't work for me.

21 08 2020 16:09:00.623:ERROR [coverage]: Chrome Headless 84.0.4147.135 (Mac OS 10.15.6): Coverage for statements (88.29%) does not meet global threshold (100%)
21 08 2020 16:09:00.623:ERROR [coverage]: Chrome Headless 84.0.4147.135 (Mac OS 10.15.6): Coverage for branches (76.12%) does not meet global threshold (100%)
21 08 2020 16:09:00.623:ERROR [coverage]: Chrome Headless 84.0.4147.135 (Mac OS 10.15.6): Coverage for lines (92.55%) does not meet global threshold (100%)
21 08 2020 16:09:00.623:ERROR [coverage]: Chrome Headless 84.0.4147.135 (Mac OS 10.15.6): Coverage for functions (94.74%) does not meet global threshold (100%)

echo $?
0

Exists with 0 instead

cat ./node_modules/karma-coverage/package.json | grep version
  "version": "2.0.3"

macOS Catalina, nodejs v12.16.3

jesusbarcenilla commented

@industral It is not going to work until karma release a new version, probably v5.1.2

industral commented

aha, got it. Thanks @jesusbarcenilla for clarifying. Well, waiting :)

matz3 commented

Solved with karma v5.2.0 (in combination with karma-coverage v2.0.3)

  • 👍1