[0.18.0] Running black causes core dump: src/unix/core.c:633: uv__close: Assertion `fd > 2' failed.
khneal opened this issue · 5 comments
khneal commented
- uvloop version: 0.18.0 (not 0.17.0)
- Python version: 3.8 slim buster, 3.11 slim bookworm
- Platform: Linux (Debian) amd64
- Can you reproduce the bug with
PYTHONASYNCIODEBUGin env?: Yes - Does uvloop behave differently from vanilla asyncio? How?: n/a
While building a Docker image with podman in Jenkins, running black causes a core dump:
STEP 14/15: RUN venv/bin/black --verbose --check --diff . || true
Identified `/` as project root containing a file system root.
Found input source directory: "/approot"
/approot/venv ignored: matches the --exclude regular expression
python: src/unix/core.c:633: uv__close: Assertion `fd > 2' failed.
Aborted (core dumped)
The error seems to refer to: https://github.com/libuv/libuv/blob/v1.46.0/src/unix/core.c#L633
I was able to reproduce with python 3.8-slim, 3.11-slim, and 3.12-slim. I did not try others.
The core dump does not happen when I switch back to 0.17.0, so we have a workaround for now.
See attached Dockerfile.
fantix commented
This is probably happening in a situation not reflected in the provided Dockerfile because I cannot reproduce this with either Podman, Docker or a local run. The link to the libuv code (thanks!) is also a very generic fd close call used in many places; thus, I'm unable to locate the issue atm.
It looks like black is using asyncio (uvloop if present) for its concurrency, so it would be great if you could try debugging the crash (a call stack would be nice!) or even better - find a reproducible Python script.
khneal commented
Producing a stacktrace from the core dump is on my to-do list... thank you for your response, and for your patience with me following up.
khneal commented
Does this help?
STEP 16/16: RUN venv/bin/pystack core --native --locals core.*
Using executable found in the core file: venv/bin/python
Core file information:
state: R zombie: True niceness: 0
pid: 1436 ppid: 1435 sid: 7
uid: 0 gid: 0 pgrp: 7
executable: python arguments: venv/bin/python -m black --verbose --check --diff .
The process died due receiving signal SIGABRT
Traceback for thread 1436 [Has the GIL] (most recent call last):
(C) File "???", line 0, in _start (python)
(C) File "???", line 0, in __libc_start_main (libc.so.6)
(C) File "???", line 0, in Py_BytesMain (libpython3.8.so.1.0)
(C) File "???", line 0, in Py_RunMain (libpython3.8.so.1.0)
(Python) File "/usr/local/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
Arguments:
alter_argv: 1
mod_name: "black.__main__"
Locals:
main_globals: {"__name__": "__main__", "__doc__": None, "__package__": "black", ...}
mod_spec: <ModuleSpec at 0x7f996f33a610>
code: <code at 0x7f996f678be0>
(Python) File "/usr/local/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
Arguments:
script_name: None
pkg_name: "black"
run_globals: {"__name__": "__main__", "__doc__": None, "__package__": "black", ...}
code: <code at 0x7f996f678be0>
init_globals: None
mod_name: "__main__"
mod_spec: <ModuleSpec at 0x7f996f33a610>
Locals:
cached: "/approot/venv/lib/python3.8/site-packages/black/__pycache__/__main__.cpython...
loader: <SourceFileLoader at 0x7f99726ae730>
fname: "/approot/venv/lib/python3.8/site-packages/black/__main__.py"
(Python) File "/approot/venv/lib/python3.8/site-packages/black/__main__.py", line 3, in <module>
patched_main()
(C) File "???", line 0, in CPyPy_black___patched_main (2ec0e72aa72355e6eccf__mypyc.cpython-38-x86_64-linux-gnu.so)
(C) File "???", line 0, in CPyDef_black___patched_main (2ec0e72aa72355e6eccf__mypyc.cpython-38-x86_64-linux-gnu.so)
(Python) File "/approot/venv/lib/python3.8/site-packages/click/core.py", line 1157, in __call__
return self.main(*args, **kwargs)
Arguments:
self: <Command at 0x7f996f33a6d0>
Locals:
kwargs: {}
args: ()
(Python) File "/approot/venv/lib/python3.8/site-packages/click/core.py", line 1078, in main
rv = self.invoke(ctx)
Arguments:
complete_var: None
windows_expand_args: True
standalone_mode: True
prog_name: "python -m black"
self: <Command at 0x7f996f33a6d0>
args: []
Locals:
ctx: <Context at 0x7f99725f8310>
extra: {}
(Python) File "/approot/venv/lib/python3.8/site-packages/click/core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
Arguments:
ctx: <Context at 0x7f99725f8310>
self: <Command at 0x7f996f33a6d0>
(Python) File "/approot/venv/lib/python3.8/site-packages/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
Arguments:
_Context__callback: <function at 0x7f99725a9430>
_Context__self: <Context at 0x7f99725f8310>
Locals:
ctx: <Context at 0x7f99725f8310>
args: ()
kwargs: {"src": ("."), "stdin_filename": None, "config": None, "verbose": True, ...}
(Python) File "/approot/venv/lib/python3.8/site-packages/click/decorators.py", line 33, in new_func
return f(get_current_context(), *args, **kwargs)
Locals:
kwargs: {"src": ("."), "stdin_filename": None, "config": None, "verbose": True, ...}
args: ()
(C) File "???", line 0, in CPyPy_black___main (2ec0e72aa72355e6eccf__mypyc.cpython-38-x86_64-linux-gnu.so)
(C) File "???", line 0, in CPyDef_black___main (2ec0e72aa72355e6eccf__mypyc.cpython-38-x86_64-linux-gnu.so)
(Python) File "/approot/venv/lib/python3.8/site-packages/black/concurrency.py", line 97, in reformat_many
loop = asyncio.new_event_loop()
Arguments:
workers: 32
report: <Report at 0x7f99726ae280>
sources: <set at 0x7f996f591ac0>
mode: < at 0x7f996f338ef0>
write_back: <WriteBack at 0x7f996f6dd8b0>
fast: False
Locals:
executor: <ProcessPoolExecutor at 0x7f996e4c3370>
(Python) File "/usr/local/lib/python3.8/asyncio/events.py", line 758, in new_event_loop
return get_event_loop_policy().new_event_loop()
(Python) File "/usr/local/lib/python3.8/asyncio/events.py", line 656, in new_event_loop
return self._loop_factory()
Arguments:
self: <EventLoopPolicy at 0x7f996f33a970>
(Python) File "/approot/venv/lib/python3.8/site-packages/uvloop/__init__.py", line 137, in _loop_factory
return new_event_loop()
Arguments:
self: <EventLoopPolicy at 0x7f996f33a970>
(Python) File "/approot/venv/lib/python3.8/site-packages/uvloop/__init__.py", line 22, in new_event_loop
return Loop()
(C) File "uvloop/loop.c", line 152942, in __pyx_tp_new_6uvloop_4loop_Loop (loop.cpython-38-x86_64-linux-gnu.so)
(C) File "uvloop/loop.c", line 10385, in __pyx_pw_6uvloop_4loop_4Loop_1__cinit__ (inlined) (loop.cpython-38-x86_64-linux-gnu.so)
(C) File "uvloop/loop.c", line 10642, in __pyx_pf_6uvloop_4loop_4Loop___cinit__ (inlined) (loop.cpython-38-x86_64-linux-gnu.so)
(C) File "src/unix/loop.c", line 80, in uv_loop_init (loop.cpython-38-x86_64-linux-gnu.so)
(C) File "src/unix/linux.c", line 590, in uv__platform_loop_init (loop.cpython-38-x86_64-linux-gnu.so)
(C) File "src/unix/linux.c", line 562, in uv__iou_init (loop.cpython-38-x86_64-linux-gnu.so)
(C) File "???", line 0, in abort (libc.so.6)
(C) File "???", line 0, in raise (libc.so.6)
nayanmanojgupta commented
Any update on this?
khneal commented
Closing... I'm now unable to reproduce the original error.