dlcov coverage value lower than the value of plain "flutter test --coverage"

[wheeOs]

I noticed a surprising discrepancy. When I run flutter test --coverage and evaluate the coverage in a classical way, the coverage value is higher than the value output from

COVERAGE_THRESH=5
EXCLUDED_SUFFIXES=.g.dart,.freezed.dart
flutter pub run dlcov -c $COVERAGE_THRESH -p dummy --lcov-gen="flutter test --coverage ./lib" --include-untested-files=true --exclude-suffix=$EXCLUDED_SUFFIXES

Even though, in my case, the classical approach also uses the generated dlcov_references_test.dart I would have expected that the value from dlcov is at least equal or higher. Why is this the case?

Please note, that in some files I used the flags // coverage:ignore-file. Maybe this detail is relevant.

• the result from dlcov:

[SUCCESS]: The total code coverage 15.0% is equal to, or greater than expected 5.0%

• the result from flutter test --coverage (presented by Flutter Enhancement Suite)

[emanuel-braz]

Hi @wheeOs , thanks for the contribution! Yes, it looks like that flag is relevante. Can you test something?

dart pub global activate coverage
and
flutter pub run dlcov -c $COVERAGE_THRESH -p dummy --lcov-gen="dart pub global run coverage:test_with_coverage" --include-untested-files=true --exclude-suffix=$EXCLUDED_SUFFIXES

I am not sure, but it looks like, only the coverage lib on pub.dev is aware of this flag // coverage:ignore-file

[wheeOs]

unfortunately that kinda doesn't work

Running "dart pub global run coverage:test_with_coverage" failed with 255.
pub finished with exit code 255

[wheeOs]

...
I am not sure, but it looks like, only the coverage lib on pub.dev is aware of this flag // coverage:ignore-file

I'm not sure about that either. Because we use gitlab's ci for example, where also a coverage check happens. We don't use the coverage package (at least not explicitly). And there we also get a result of 19%.

a snippet of the ci script:

    - lcov -r coverage/lcov.info '*/__test*__/*' -o coverage/lcov_cleaned.info
    - genhtml coverage/lcov_cleaned.info --output=coverage

[emanuel-braz]

I guess the problem is the lcov file generation step.

Do you have this flutter pub global activate junitreport command and/or flutter test --machine | tojunit -o report.xml into gitlab-ci.yaml file?

I was looking for what GitLab uses and find a post about configuring GitLab. And a found this junitreport.
If yes, I can say junitreport - tojunit uses coverage as transitive dependency.

You can check that flutter test --coverage include files with flag // coverage:ignore-file into the generated lcov file coverage/lcov.info, but coverage package, does not.

There is a parameter called exclude-contents in dlcov.
This command below, ignores files with the flag // coverage:ignore-file(as coverage does), can you try it?

flutter pub run dlcov -c $COVERAGE_THRESH -p dummy --lcov-gen="flutter test --coverage ./lib" --include-untested-files=true --exclude-suffix=$EXCLUDED_SUFFIXES --exclude-contents="//*coverage:ignore-file"

[wheeOs]

I eagerly want to check your suggested command.. but unfortunately as soon as I add the argument --exclude-contents="//*coverage:ignore-file" the test process just doesn't finish and hangs up on one of our tests which use async closures.
When I omit the --exclude-contents part, the tests run just fine.

[wheeOs]

finally I got it working. Due to a typo in the name for -p the generated file had errors (which script command line also used --exclude-contents), but --exclude-contents="//*coverage:ignore-file" haven't changed the result. It's still at 15%