pytest annoyances and intermittent segfault
jcermauwedu opened this issue · 4 comments
jcermauwedu commented
Most tests apparently pass. With python 3.11.9, there are these floating around.
Error in sys.excepthook:
Original exception was:
Error in sys.excepthook:
Original exception was:
Error in sys.excepthook:
Original exception was:
Error in sys.excepthook:
Original exception was:
Error in sys.excepthook:
Original exception was:
Error in sys.excepthook:
Original exception was:
Error in sys.excepthook:
Original exception was:
Error in sys.excepthook:
Original exception was:
Intermittently, I can trigger a segfault...
$ pytest --pdb
============================================================================================= test session starts =============================================================================================
platform linux -- Python 3.11.9, pytest-8.2.0, pluggy-1.5.0
rootdir: /home/portal/src/compliance-checker
configfile: pyproject.toml
plugins: requests-mock-1.12.1, time-machine-2.14.1
collected 235 items
compliance_checker/tests/test_acdd.py ................ [ 6%]
compliance_checker/tests/test_base.py ........ [ 10%]
compliance_checker/tests/test_cf.py ........................................................................................... [ 48%]
compliance_checker/tests/test_cf_integration.py ................. [ 56%]
compliance_checker/tests/test_cli.py ......... [ 60%]
compliance_checker/tests/test_feature_detection.py ............................... [ 73%]
compliance_checker/tests/test_ioos_profile.py .......................................... [ 91%]
compliance_checker/tests/test_ioos_sos.py .. [ 91%]
compliance_checker/tests/test_protocols.py ....s [ 94%]
compliance_checker/tests/test_suite.py ............Fatal Python error: Segmentation fault
Current thread 0x0000784bf591e740 (most recent call first):
File "/home/portal/src/compliance-checker/compliance_checker/cf/cf_1_6.py", line 749 in _check_valid_cf_units
File "/home/portal/src/compliance-checker/compliance_checker/cf/cf_1_6.py", line 707 in check_units
File "/home/portal/src/compliance-checker/compliance_checker/suite.py", line 241 in _run_check
File "/home/portal/src/compliance-checker/compliance_checker/suite.py", line 421 in run_all
File "/home/portal/src/compliance-checker/compliance_checker/tests/test_suite.py", line 66 in test_unicode_formatting
File "/home/portal/miniconda3/envs/gutils_py311/lib/python3.11/unittest/case.py", line 579 in _callTestMethod
File "/home/portal/miniconda3/envs/gutils_py311/lib/python3.11/unittest/case.py", line 623 in run
File "/home/portal/miniconda3/envs/gutils_py311/lib/python3.11/unittest/case.py", line 678 in __call__
File "/home/portal/miniconda3/envs/gutils_py311/lib/python3.11/site-packages/_pytest/unittest.py", line 343 in runtest
File "/home/portal/miniconda3/envs/gutils_py311/lib/python3.11/site-packages/_pytest/runner.py", line 173 in pytest_runtest_call
File "/home/portal/miniconda3/envs/gutils_py311/lib/python3.11/site-packages/pluggy/_callers.py", line 103 in _multicall
File "/home/portal/miniconda3/envs/gutils_py311/lib/python3.11/site-packages/pluggy/_manager.py", line 120 in _hookexec
File "/home/portal/miniconda3/envs/gutils_py311/lib/python3.11/site-packages/pluggy/_hooks.py", line 513 in __call__
File "/home/portal/miniconda3/envs/gutils_py311/lib/python3.11/site-packages/_pytest/runner.py", line 241 in <lambda>
File "/home/portal/miniconda3/envs/gutils_py311/lib/python3.11/site-packages/_pytest/runner.py", line 341 in from_call
File "/home/portal/miniconda3/envs/gutils_py311/lib/python3.11/site-packages/_pytest/runner.py", line 240 in call_and_report
File "/home/portal/miniconda3/envs/gutils_py311/lib/python3.11/site-packages/_pytest/runner.py", line 135 in runtestprotocol
File "/home/portal/miniconda3/envs/gutils_py311/lib/python3.11/site-packages/_pytest/runner.py", line 116 in pytest_runtest_protocol
File "/home/portal/miniconda3/envs/gutils_py311/lib/python3.11/site-packages/pluggy/_callers.py", line 103 in _multicall
File "/home/portal/miniconda3/envs/gutils_py311/lib/python3.11/site-packages/pluggy/_manager.py", line 120 in _hookexec
File "/home/portal/miniconda3/envs/gutils_py311/lib/python3.11/site-packages/pluggy/_hooks.py", line 513 in __call__
File "/home/portal/miniconda3/envs/gutils_py311/lib/python3.11/site-packages/_pytest/main.py", line 364 in pytest_runtestloop
File "/home/portal/miniconda3/envs/gutils_py311/lib/python3.11/site-packages/pluggy/_callers.py", line 103 in _multicall
File "/home/portal/miniconda3/envs/gutils_py311/lib/python3.11/site-packages/pluggy/_manager.py", line 120 in _hookexec
File "/home/portal/miniconda3/envs/gutils_py311/lib/python3.11/site-packages/pluggy/_hooks.py", line 513 in __call__
File "/home/portal/miniconda3/envs/gutils_py311/lib/python3.11/site-packages/_pytest/main.py", line 339 in _main
File "/home/portal/miniconda3/envs/gutils_py311/lib/python3.11/site-packages/_pytest/main.py", line 285 in wrap_session
File "/home/portal/miniconda3/envs/gutils_py311/lib/python3.11/site-packages/_pytest/main.py", line 332 in pytest_cmdline_main
File "/home/portal/miniconda3/envs/gutils_py311/lib/python3.11/site-packages/pluggy/_callers.py", line 103 in _multicall
File "/home/portal/miniconda3/envs/gutils_py311/lib/python3.11/site-packages/pluggy/_manager.py", line 120 in _hookexec
File "/home/portal/miniconda3/envs/gutils_py311/lib/python3.11/site-packages/pluggy/_hooks.py", line 513 in __call__
File "/home/portal/miniconda3/envs/gutils_py311/lib/python3.11/site-packages/_pytest/config/__init__.py", line 178 in main
File "/home/portal/miniconda3/envs/gutils_py311/lib/python3.11/site-packages/_pytest/config/__init__.py", line 206 in console_main
File "/home/portal/miniconda3/envs/gutils_py311/bin/pytest", line 10 in <module>
Extension modules: _brotli, simplejson._speedups, _time_machine, numpy.core._multiarray_umath, numpy.core._multiarray_tests, numpy.linalg._umath_linalg, numpy.fft._pocketfft_internal, numpy.random._common, numpy.random.bit_generator, numpy.random._bounded_integers, numpy.random._mt19937, numpy.random.mtrand, numpy.random._philox, numpy.random._pcg64, numpy.random._sfc64, numpy.random._generator, cftime._cftime, netCDF4._netCDF4, cf_units._udunits2, lxml._elementpath, lxml.etree, regex._regex, pyproj._compat, pyproj._datadir, pyproj._network, pyproj._geod, pyproj.list, pyproj._crs, pyproj.database, pyproj._transformer, pyproj._sync, shapely.lib, shapely._geos, shapely._geometry_helpers, markupsafe._speedups (total: 35)
Segmentation fault (core dumped)
benjwadams commented
I tried fixing these by suppressing the destructor in some previous commits. I've been unable to determine how exactly these are being called in such a way where I can stop these errors from appearing on the CI tests.
ocefpaf commented
I tried fixing these by suppressing the destructor in some previous commits. I've been unable to determine how exactly these are being called in such a way where I can stop these errors from appearing on the CI tests.
Most of them are due to caching on top of caching that are not release. Some, if not all, are not needed b/c the underlying slow/"called multiple times" function is already cached upstream. I'd love a speed test to be 100% sure we are not losing speed though. See #1069.
There are a few others due to how the mocked netcdf Dataset is built. I've been unable to get rid of those. Investigating more... But you can see them in action when you call self.cf.setup in the tests.
ocefpaf commented
@jcermauwedu I never experienced a segfault like that but do you mind testing against #1069 to see if that solves your problem?
jcermauwedu commented
A stress test yielded no more segfaults. If another one appears, we will let you know.
Using the latest commit, the output is squeaky clean. Thanks @benjwadams @ocefpaf
$ pytest
======================================================================== test session starts =========================================================================
platform linux -- Python 3.11.9, pytest-8.2.0, pluggy-1.5.0
rootdir: /home/portal/src/compliance-checker
configfile: pyproject.toml
plugins: cov-5.0.0, requests-mock-1.12.1, vcr-1.0.2, time-machine-2.14.1
collected 235 items
compliance_checker/tests/test_acdd.py ................ [ 6%]
compliance_checker/tests/test_base.py ........ [ 10%]
compliance_checker/tests/test_cf.py ........................................................................................... [ 48%]
compliance_checker/tests/test_cf_integration.py ................. [ 56%]
compliance_checker/tests/test_cli.py ......... [ 60%]
compliance_checker/tests/test_feature_detection.py ............................... [ 73%]
compliance_checker/tests/test_ioos_profile.py .......................................... [ 91%]
compliance_checker/tests/test_ioos_sos.py .. [ 91%]
compliance_checker/tests/test_protocols.py ....s [ 94%]
compliance_checker/tests/test_suite.py ............. [ 99%]
compliance_checker/tests/test_util.py . [100%]
============================================================= 234 passed, 1 skipped in 70.58s (0:01:10) ==============================================================