Python 3.11: comments / docstrings / future imports at the beginning of code cells cause crashes
jerry-git opened this issue · 9 comments
Versions: Python 3.11 and ipytest==0.13.1.
Comments at the beginning of code cells cause crashes after ipytest.autoconfig() has been executed. Here's a simple example:
FWIW, I tried running the ipytest test suite with 3.11, no failures.
Hey @jerry-git Thanks a lot for the report.
I can reproduce the issue. This is somehow connected to pytest's assertion rewriting. I will have to figure out where the error comes from.
Until I do, you can disable assertion rewriting. While this will disable the nice error messages, at least there won't be any errors. You can change the config via:
import ipytest
ipytest.autoconfig(rewrite_asserts=False)Explanation:
- The assert rewriter does not include end_lineno for its generated AST nodes
- Ipytest calls fix_missing_locations which copies in an invalid range for the location (source)
- The Python compiler fails
Related PyTest issue:
Simple repo:
import ast
from _pytest.assertion.rewrite import rewrite_asserts
code = """# hello
a = 2
"""
mod = ast.parse(code)
rewrite_asserts(mod, b"")
mod = ast.fix_missing_locations(mod)
compile(mod, "foo.py", "exec")
# ValueError: AST node line range (2, 1) is not valid@jerry-git FYI, when I execute the integration tests locally with python 3.11, I get a test failure:
python x.py precommit
:: python.exe -m black ipytest tests Example.ipynb x.py minidoc.py
All done! ✨ 🍰 ✨
19 files left unchanged.
:: python.exe -m ruff .
Update documentation
:: python.exe -m pytest tests
0.00s - Debugger warning: It seems that frozen modules are being used, which may
0.00s - make the debugger miss breakpoints. Please pass -Xfrozen_modules=off
0.00s - to python to disable frozen modules.
0.00s - Note: Debugging will proceed. Set PYDEVD_DISABLE_FILE_VALIDATION=1 to disable this validation.
================================================================================================================================= test session starts =================================================================================================================================
platform win32 -- Python 3.11.3, pytest-7.3.1, pluggy-1.0.0
plugins: anyio-3.6.2, nbval-0.10.0, asyncio-0.21.0
asyncio: mode=Mode.STRICT
collected 67 items
tests\test_config.py .. [ 2%]
tests\test_doctest.py .. [ 5%]
tests\test_force_reload.py .... [ 11%]
tests\test_ipytest.py ........................................... [ 76%]
tests\test_issue_71.py .......... [ 91%]
tests\test_raise_on_error.py ...... [100%]
================================================================================================================================= 67 passed in 2.48s ==================================================================================================================================
:: python.exe -m pytest --nbval-lax --current-env Example.ipynb
0.00s - Debugger warning: It seems that frozen modules are being used, which may
0.00s - make the debugger miss breakpoints. Please pass -Xfrozen_modules=off
0.00s - to python to disable frozen modules.
0.00s - Note: Debugging will proceed. Set PYDEVD_DISABLE_FILE_VALIDATION=1 to disable this validation.
================================================================================================================================= test session starts =================================================================================================================================
platform win32 -- Python 3.11.3, pytest-7.3.1, pluggy-1.0.0
plugins: anyio-3.6.2, nbval-0.10.0, asyncio-0.21.0
asyncio: mode=Mode.STRICT
collected 11 items
Example.ipynb ..F.....s.. [100%]
====================================================================================================================================== FAILURES =======================================================================================================================================
________________________________________________________________________________________________________________________________ Example.ipynb::Cell 2 ________________________________________________________________________________________________________________________________
Notebook cell execution failed
Cell 2: Cell execution caused an exception
Input:
%%ipytest
# define the tests
def test_my_func():
assert my_func(0) == 0
assert my_func(1) == 0
assert my_func(2) == 2
assert my_func(3) == 2
def my_func(x):
return x // 2 * 2
Traceback:
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
File ~\miniconda3\envs\py311\Lib\codeop.py:118, in Compile.__call__(self, source, filename, symbol)
117 def __call__(self, source, filename, symbol):
--> 118 codeob = compile(source, filename, symbol, self.flags, True)
119 for feature in _features:
120 if codeob.co_flags & feature.compiler_flag:
ValueError: AST node line range (3, 1) is not valid
@jerry-git I released ipytest==0.13.2b0 with a fix. Would be cool, if you could have look, whether this solves your issue. I will also test it a for a bit and then do a proper 0.13.2 release.
@chmp Appreciate the heads-up over at the pytest repo, and the quick fix! I use ipytest for company trainings/talks about pytest, and I bet this would have bitten me with a live demo-heavy talk next week 😅
@The-Compiler thanks for the feedback. Always nice to hear that someone is using your stuff :) Not sure I will manage a proper release until next week, so hopefully the pre-release works for you. I would be interested to hear whether your content works with the fix without any issues.
when I execute the integration tests locally with python 3.11, I get a test failure ...
That's great! I guess I didn't run the integration tests, just the unit ones.
Would be cool, if you could have look, whether this solves your issue.
Yup, works 👍
Thanks for the swift reaction 🙏
Thanks for the feedback. Glad everything works.
Released as v0.13.2