Coverage wrongly calculated

[t-pohl]

Summary

When using pytest --cov together with a FastAPI app, sqlalchemy (async Postgres connection) and pytest-docker (for the DB) we get back wrongly calculated coverage for the services. For organization purposes we separate our functionality into routers and services where the routers import and use the services.

Expected vs actual result

While the coverage in our router functions seems to be right the coverage in our service functions is not large enough i.e. some lines which are definitely covered are marked as uncovered.

We already tested the following possible fixes:

• eager loading of the pytest-cov plugin via -p
• explicitly setting the coverage sources in the command via --cov=...
• Having __init__.py in all subdirectories

Reproducer

Versions

Python: 3.11.2
pytest: 7.3.1

Config

pyproject.toml:

[tool.coverage.run]
branch = true
source = ['backend']
omit = [
    '*__init__.py',
    'backend/api/schemas/*',
    'backend/database/*',
    'backend/application.py',
    'backend/settings.py',
]

Code

Repo: https://github.com/t-pohl/pytest-coverage-err

Just clone the repo, run poetry install and then poetry run scripts/coverage.sh and observe the results.

The asset_service.py file should have a higher coverage value. One example would be lines 39-41 which are reported as uncovered but are definitely called in the test test_get_asset_not_found inside the file tests/api/test_03_asset.py.

[etr2460]

Paying this forward, I took a trip through multiple github issues/repos and finally found nedbat/coveragepy#841 (comment) which fixed coverage for projects that used async sqlalchemy. Adding:

[tool.coverage.run]
concurrency = ["greenlet"]

to my pyproject.toml did the trick.

This was also the fix for nedbat/coveragepy#1216, which is a similar instance of a few libraries interacting in unexpected ways to break coverage. I hope this helps people find this fix in the future!