pytest-dev/pytest-github-actions-annotate-failures

Internal error in `pytest` when using `pytest-github-actions-annotate-failures` v0.3.0

ganow opened this issue · 5 comments

ganow commented

Thank you for developing a great piece of software. I am using pytest-github-actions-annotate-failures in our GitHub Actions workflow.

Recently, I encountered an issue where a previously working workflow stopped functioning due to an internal error in pytest. After investigating, I found that this issue only occurs only when pytest-github-actions-annotate-failures v0.3.0 is installed, and it does not happen with v0.2.0.

Note that I have determined that this issue occurs only when running a specific test case within the repository we are developing. However, I have not yet identified the exact part of the codebase that triggers the internal error. As a result, I have not been able to create a minimal reproducible example from the codebase.

This issue is causing me difficulties, and I would really appreciate any insights or suggestions that could help resolve it. Below is the error message that appears when using v0.3.0. If there is any additional information I can provide to help diagnose the issue, please let me know. I would also appreciate it if you could let me know if there is any issue with how I am using the plugin. Thank you.

Environment

  • Testing Setup: The issue is reproducible both on GitHub Actions and in a local environment using act
  • Versions:
    • Python: 3.8.18, 3.9.12, 3.10.16, 3.11.11
    • pytest: 8.3.4
    • pluggy: 1.5.0
    • pytest-github-actions-annotate-failures: 0.3.0

(Minimal) workflow file

on:
  push:
    branches: [ "dev" ]
  pull_request:
    branches: [ "dev" ]

jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false
      matrix:
        python-version: ["3.8", "3.9", "3.10", "3.11"]

    steps:
    - uses: actions/checkout@v3

    - name: Set up Python ${{ matrix.python-version }}
      uses: actions/setup-python@v4
      with:
        python-version: ${{ matrix.python-version }}
        cache: "pip"

    - name: Install dependencies
      run: |
        python -m pip install --upgrade pip
        python -m pip install pytest pytest-cov pytest-github-actions-annotate-failures
        pip install .

    - name: Test with pytest
      run: pytest

Error message

INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/_pytest/main.py", line 283, in wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>   File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/_pytest/main.py", line 337, in _main
INTERNALERROR>     config.hook.pytest_runtestloop(session=session)
INTERNALERROR>   File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/pluggy/_hooks.py", line 513, in __call__
INTERNALERROR>     return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
INTERNALERROR>   File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/pluggy/_manager.py", line 120, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR>   File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/pluggy/_callers.py", line 156, in _multicall
INTERNALERROR>     teardown[0].send(outcome)
INTERNALERROR>   File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/pluggy/_result.py", line 100, in get_result
INTERNALERROR>     raise exc.with_traceback(exc.__traceback__)
INTERNALERROR>   File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/pluggy/_callers.py", line 156, in _multicall
INTERNALERROR>     teardown[0].send(outcome)
INTERNALERROR>   File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/_pytest/logging.py", line 803, in pytest_runtestloop
INTERNALERROR>     return (yield)  # Run all the tests.
INTERNALERROR>   File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/pluggy/_callers.py", line 156, in _multicall
INTERNALERROR>     teardown[0].send(outcome)
INTERNALERROR>   File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/_pytest/terminal.py", line 673, in pytest_runtestloop
INTERNALERROR>     result = yield
INTERNALERROR>   File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/pluggy/_callers.py", line 103, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/_pytest/main.py", line 362, in pytest_runtestloop
INTERNALERROR>     item.config.hook.pytest_runtest_protocol(item=item, nextitem=nextitem)
INTERNALERROR>   File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/pluggy/_hooks.py", line 513, in __call__
INTERNALERROR>     return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
INTERNALERROR>   File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/pluggy/_manager.py", line 120, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR>   File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/pluggy/_callers.py", line [139](https://github.com/KamitaniLab/bdpy/actions/runs/13387656283/job/37387894787?pr=101#step:5:140), in _multicall
INTERNALERROR>     raise exception.with_traceback(exception.__traceback__)
INTERNALERROR>   File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/pluggy/_callers.py", line 124, in _multicall
INTERNALERROR>     teardown.send(result)  # type: ignore[union-attr]
INTERNALERROR>   File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/_pytest/warnings.py", line 112, in pytest_runtest_protocol
INTERNALERROR>     return (yield)
INTERNALERROR>   File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/contextlib.py", line 120, in __exit__
INTERNALERROR>     next(self.gen)
INTERNALERROR>   File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/_pytest/warnings.py", line 67, in catch_warnings_for_item
INTERNALERROR>     ihook.pytest_warning_recorded.call_historic(
INTERNALERROR>   File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/pluggy/_hooks.py", line 535, in call_historic
INTERNALERROR>     res = self._hookexec(self.name, self._hookimpls.copy(), kwargs, False)
INTERNALERROR>   File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/pluggy/_manager.py", line 120, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR>   File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/pluggy/_callers.py", line 139, in _multicall
INTERNALERROR>     raise exception.with_traceback(exception.__traceback__)
INTERNALERROR>   File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/pluggy/_callers.py", line 103, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/pytest_github_actions_annotate_failures/plugin.py", line 127, in pytest_warning_recorded
INTERNALERROR>     message=warning_message.message.args[0],
INTERNALERROR> IndexError: tuple index out of range
edgarrmondragon commented

Hey @ganow, thanks for reporting!

In the meantime, can you try running pytest with the --exclude-warning-annotations option?

     - name: Test with pytest
-      run: pytest
+      run: pytest --exclude-warning-annotations
ganow commented

Hi @edgarrmondragon! Thank you for a quick response!

can you try running pytest with the --exclude-warning-annotations option?

I'm sorry for not mentioning this earlier, but I actually tried a similar option before. When I ran pytest with the -p no:warnings option, I didn't encounter the internal error even I used v0.3.0.

     - name: Test with pytest
-      run: pytest
+      run: pytest -p no:warnings

Should I check the --exclude-warning-annotations option, too?

Updated: I didn't see the internal error when I ran pytest with the --exclude-warning-annotations option (tested on my local laptop with act, Python version to be tested: 3.11.11)

fschulze commented

I'm getting a similar error, but with this:

INTERNALERROR>   File "/home/runner/work/devpi/devpi/postgresql/.tox/py39-server680/lib/python3.9/site-packages/pytest_github_actions_annotate_failures/plugin.py", line 127, in pytest_warning_recorded
INTERNALERROR>     message=warning_message.message.args[0],
INTERNALERROR> AttributeError: 'str' object has no attribute 'args'

I'm still investigating what is causing it.

fschulze commented

My code under test uses pluggy and has deprecated hooks (see https://pluggy.readthedocs.io/en/latest/index.html#warnings-on-hook-implementation). Those deprecation warnings seem to cause warning_message.message to be a str in pytest somehow.

fschulze commented

Looking at the warnings.py source it seems like it always converts the message to a string. So I think the code here should be:

message=str(warning_message.message)

instead of accessing args[0].