Adding ptvsd breaks pytest coverage

[mneil]

Environment data

• PTVSD version: 4.3.2
• OS and version: python:3 (debian buster) Container
• Python version (& distribution if applicable, e.g. Anaconda): 3.8.1
• Using VS Code or Visual Studio: VSCode

Actual behavior

Coverage report reads 0% covered. Html output shows 0 coverage

Expected behavior

89.92% covered

Steps to reproduce:

1. Create a simple hello world python application
2. Add a single test in tests/test_hello.py
3. Install pytest, pytest-cov pip install pytest pytest-cov
4. Run pytest with coverage, get {n}% coverage > 0. python -m pytest --cov-report term-missing -cov=$(PROJECT) tests
5. Install ptvsd pip install ptvsd
6. Import ptvsd to tests/conftest.py (don't even need to use it, just import it)
7. Run pytest again and get 0% coverage

I'm not sure if this is an issue with ptvsd or pytest. I've experience issues with pytest cov in the past and will work on running coverage separate from pytest.

[int19h]

This is sort of by design - ptvsd and codecov use the same Python tracing hook for different purposes, and so whichever one gets hold of it last is the one that works.

[mneil]

Ok, so maybe there's a better way to do what I'm hoping to do. My setup is vscode on my host machine, my application is mounted into a running container (I can't use vscode remote for various reasons) and I want to attach the debugger sometimes, but I want to unit test and check coverage quite often.

My conftest can be as simple as:

import ptvsd

if os.getenv('DEBUGGER'):
    ptvsd.enable_attach(address = ('0.0.0.0', 5678))
    ptvsd.wait_for_attach()

Where I set DEBUGGER to anything in my terminal running in the container when I want to attach the debugger. I run my unit tests via make with:

python -m pytest -vvv \
    -W ignore::DeprecationWarning \
    --cov-report html \
    --cov-report term-missing \
    --cov=$(PROJECT) \
    --cov-fail-under=80 \
    tests/unit$(target)

Where I might run target=/pythonfile.py::single_test make unit to run a single test or simply make unit to run the full suite of unit tests.

I guess I could get rid of the DEBUGGER flag and instead add a new make target that executes tests under ptvsd? Any suggestions?

[mneil]

I guess I'm not sure how to use ptvsd from the cli with pytest. Running

debug:
    python -m ptvsd --host 0.0.0.0  --port 5678 --wait \
        pytest -vvv \
        tests/unit$(target)

Gives me an error when I attach the debugger, No such file or directory: 'pytest' which makes sense - because ptvsd is looking for a module to target but is getting pytest instead.

[int19h]

Just like with Python itself, you need to use -m pytest to indicate that the command line argument is a module rather than a filename.

[mneil]

That works. I'm about a year into writing python so it wasn't immediately obvious to me based on the documentation that I'd need -m twice. In my mind, the -m prefixed in all calls in the readme was the typical python -m and I thought you just left out the python part. The help docs state -m <module>, which again, I confused with python module. This probably makes perfect sense if you work on this project and now, I'm assuming, that ptvsd relaunches the module as a python sub processes.

python -m ptvsd --host 0.0.0.0  --port 5678 --wait \
    -m pytest -vvv \
    tests/unit$(target)

Anyway, this works well enough for my use case and I'm happy to be able to attach the debugger again without much effort to my dev workflow. Thanks!