pytest-cov reports uncovered if using SystemExit in an Exception block

[cochrasc]

Summary

I have a pytest test that is working as expected.
The issue is that since SystemExit is derived from BaseException instead of Exception, code coverage doesn't count the Exception as being covered. If I change "Exception" to "BaseException", it's counted properly, but using BaseException is considered a "no-no" from what I've read.

Expected vs actual result

Actual Result: Exception block is not marked as covered if using SystemExit inside of it
Expected Result: Exception block should be covered if using SystemExit inside of it

Reproducer

Versions

[tool.poetry.dependencies]
python = "~3.11"
loguru = "^0.7.2"
tenacity = "^8.5.0"
requests = "^2.32.3"
azure-mgmt-dns = "^8.1.0"
azure-core = "^1.30.0"
azure-common = "^1.1.28"
azure-identity = "^1.15.0"
azure-mgmt-resource = "^23.0.1"
azure-mgmt-privatedns = "^1.1.0"
azure-mgmt-subscription = "^3.1.1"
azure-mgmt-network = "^26.0.0"

[tool.poetry.group.dev.dependencies]
git-cliff = "^2.3.0"
pre-commit = "^3.7.0"
rich = "^13.7.1"
ruff = "^0.6.1"
pytest = "^8.2.2"
pytest-cov = "^5.0.0"
pytest-mock = "^3.14.0"
mypy = "^1.10.1"
types-requests = "^2.32.0"
responses = "^0.25.3"
python-dotenv = "^1.0.1"
pytest-loguru = "^0.4.0"

Config

[tool.coverage.run]
source = ["src"]
omit = ["tests/*"]
branch = false

[tool.coverage.report]
exclude_lines = [
# Don't complain if non-runnable code isn't run:
"if name == .main.:",
]
fail_under = 20

Code

cmd_args = []
with pytest.raises(SystemExit) as exp:
  get_args(cmd_args)
assert exp.type == SystemExit
assert exp.value.code == 2

My actual python snippet is:

except Exception as ex:
  logger.exception(ex)
  raise SystemExit(3) from ex

[nedbat]

I don't understand the details here. Can you post a complete runnable example of the code that isn't counting the line properly, and then also complete code where it does count it properly?

[cochrasc]

Test to check exception:

def test_get_args_exit() -> None:
    """Test that argparse exits when encountering an exception."""
    cmd_args = []
    with pytest.raises(SystemExit) as exp:
        get_args(cmd_args)
    assert exp.type == SystemExit
    assert exp.value.code == 2

Code that is tested against and is counting coverage correctly:

def get_args(arg_list: list[str] | None):
    """Get and parse command line arguments.
    Returns:
        argparse.Namespace:  the command line arguments and their values
    """
    try:
        parser = argparse.ArgumentParser(
            description=("Python project to create a vnet peer to the correct firewall for the subscription."),
            formatter_class=argparse.RawTextHelpFormatter,
        )
        parser.add_argument(
            "-c", "--use_cli_cred", action="store_true", help="Flag to set if using azure cli credentials."
        )
        parser.add_argument("-e", "--environment", required=True)
        parser.add_argument("-r", "--location", required=True)
        parser.add_argument("-g", "--vnet_rg", required=True)
        parser.add_argument("-n", "--vnet_name", required=True)
        parser.add_argument("-i", "--subscription_id", required=True)
        return MyProgramArgs(**vars(parser.parse_args(arg_list)))
    except BaseException as ex:
        logger.exception(ex)
        raise SystemExit(3) from ex

Not working example:
Exact same code as above, but replace "except BaseException as ex:" with "except Exception as ex:"

If I use BaseException with SystemExit, coverage counts the lines as covered, but many articles say using BaseException in this way is a bad idea
If I use Exception with SystemExit, coverage does NOT count the lines as covered:

---------- coverage: platform linux, python 3.11.9-final-0 -----------
Name              Stmts   Miss  Cover   Missing
-----------------------------------------------
src/__init__.py       0      0   100%
src/__main__.py      72     37    49%   90-91, 96-108, 113-120, 169-243
src/log.py           25      0   100%
-----------------------------------------------
TOTAL                97     37    62%

Note that the code above is from src/main.py and the lines 90-91 are:
90 logger.exception(ex)
91 raise SystemExit(3) from ex

I'm assuming it's because SystemExit is a direct child of BaseException, not Exception, but that's just a guess.

I hope this makes a bit more sense.
It's entirely possible that I'm doing something inherently wrong by using SystemExit in this way instead of return, but this is how many google searches said it should be done.

[nedbat]

Thanks.

with except Exception as ex:, lines 90 and 91 are not being run, because SystemExit is not an instance of Exception, so the except clause is not run. Your analysis is correct. If you want to log an exception when the arguments to your CLI are wrong, then you will need to catch BaseException.

[nedbat]

Coverage is behaving correctly here, so I will close the issue.