Allow custom coverage on empty coverage file

[andreilg]

Is your feature request related to a problem? Please describe.
When I create new repositories, I often don't have any code to test but I like to still have my CI setup. This results in me failing the dart --test coverage=coverage part of my CI. My solution is to now create new repositories with a test/test.dart file with an empty test in it (please let me know if you believe there is a better way of handling this). However, this generates an empty lcov.info file which results in the following error from very_good_coverage: Error: lcov is empty!

Describe the solution you'd like
I'd like it if there was an option in this action to set the desired coverage amount for an empty coverage file. It could be an input with the following syntax:

empty_file_coverage: 100

Describe alternatives you've considered
The only alternative I can think of is creating actual fake tests (as opposed to the empty one I'm using) which I believe is bad practice.

Additional context
This is the full testing section of my CI

- name: Run tests with coverage
run: dart test --coverage=coverage && dart run coverage:format_coverage --packages=.dart_tool/package_config.json --lcov --in=coverage --out=coverage/lcov.info --report-on=lib

- name: Validate 100% coverage
uses: VeryGoodOpenSource/very_good_coverage@v1.2.1

Related Issues
#95 is a similar issue but wants 0% coverage on an empty file instead of 100%. Being able to set a custom amount for an empty coverage file seems to be useful for more situations than just my own.

[alestiago]

@andreilg thanks for the suggestion and sorry for the late reply.

By definition an empty coverage means 0% test coverage. This assumes that there is code to test. If there is no code to test, then empty coverage might mean 100% coverage. Perhaps, a possible solution to this edge scenario is to only throw an error when a lcov is empty and there is code to test against.

Please let me know what you think about the suggested approach.

[Deleted user]

That could work. Really the issue is simply that there is no real reason to fail a coverage test when there isn't even any code to cover yet.

[alestiago]

Thanks for the reply @andreilg.

One could also argue that there is also no real reason to run coverage CI on an empty codebase. However, I see the point when you're setting everything up.

I'll share this with the team next week and will try to get back to you. Again, thanks for opening the issue 💙 !

[alestiago]

Hi @andreilg. Thanks for waiting.

Detecting when a codebase is empty or not is complicated, since the workflow is not intended do be used under a specific file structure and hence what categorises an empty codebase may be subjective. To this day, the workflow only cares if a coverage file (eg lcov.info) is present and then proceeds to parse it and calculate the coverage.

A possible solution may be to modify the action and stop reporting errors whenever the lcov file is non existent or empty. However, an empty or non existent file doesn't imply that the codebase is empty, instead it implies either that there are no tests or that the coverage file has not yet been generated. Some users might want the workflow to throw the error to allow debugging these issues, making this option slightly unviable.

Having said this a recommended solution for your particular problem is to add a conditional statement to the coverage step, making it only ever run if an lcov file exists and it is not empty. Below you can find a snippet of code showing how to do so, and here you can see a real working example.

jobs:
  coverage:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - name: Check for existing coverage file
        id: test_coverage_file
        run: if [ -s "./coverage/lcov.info" ]; then echo "result=true" >> $GITHUB_OUTPUT ; else echo "result=false" >> $GITHUB_OUTPUT; fi
      - name: Very Good Coverage
        if: steps.test_coverage_file.outputs.result == 'true'
        uses: VeryGoodOpenSource/very_good_coverage@v2
        with:
          path: "./coverage/lcov.info"
          min_coverage: 100

Note: you should replace "./coverage/lcov.info" to the path where your coverage file is located.

Please let me know if the above helps so we can close the issue. If it doesn't help let me know to 😄 ! Again, thank you for opening the issue 💙 !

[Deleted user]

That helps, thank you.