Mark partially evaluated if conditions are not covered
nedbat opened this issue · 6 comments
Originally reported by Marc Schlaich (Bitbucket: schlamar, GitHub: schlamar)
This is a follow up from my comment in #198.
Consider the case:
#!python
def test(a, b):
if a or b:
print 1
If you run this it counts as covered if either a or b is True. But I think for a fully covered test you should consider both cases (True, False) and (False, True) before it is actually correctly covered. b could be a complicated statement which is never checked in unit tests even if this line has code coverage. Even a bug would get 100% coverage:
#!python
def test(a, b):
if a or failing_statement:
print 1
Thanks, this feature has been requested a few times, but will be very difficult to implement, if possible at all. Here's a blog post from almost six years ago explaining a possible technique: http://nedbatchelder.com/blog/200804/wicked_hack_python_bytecode_tracing.html
Another possible approach is code rewrite.
if a or b:
...Can be rewritten as:
def body():
...
if a and b:
body() # body covered via both a and b
elif a and not b:
body() # body covered via a
elif not a and b:
body() # body covered via b
else
... # body not covered at all.Since @nedbat's blogpost two things have happened (or will happen) in CPython:
- opcode tracing became a supported feature (by setting
frame.f_trace_opcodestoTrue) - code objects contain line and character range information per bytecode instruction via the new
co_positions()method (this is not yet in a stable release, but is in the latest beta for 3.11)
I've written a small proof-of-concept that shows that this could, in theory, now be done: https://gist.github.com/L3viathan/9be572bbf6659da5b9ae8af023ab3713
However, there's many obstacles in the way, and doing this would probably be quite brittle and require adjustments any time bytecode instructions or even just the way code is compiled is changed.