multidict 4.7.1 causes segfault when bundled with PyInstaller
gmarull opened this issue ยท 11 comments
Describe the bug
After upgrading to multidict
4.7.1 we have found that the applications we bundle using PyInstaller crash. Actually, I discovered this problem because we use aiohttp
which depends on multidict
. Version 4.6.1 does not have this problem.
To Reproduce
- Create a new virtual environment
pip install multidict==4.7.1 pyinstaller
- Create a simple
test.py
script with:
print("Hello!")
import multidict
- Bundle it
pyinstaller test.py
- Run it:
./dist/test/test
Hello!
[1] 22548 segmentation fault (core dumped) ./dist/test/test
Expected behavior
Application prints "Hello!" and quits nicely.
Logs/tracebacks
Provided above.
Your version of the Python
Observed with 3.6.9 and 3.7.5
Your version of the multidict distribution
4.7.1
Additional context
N/A
Im getting this with every version > 4.6.1 (4.7.0 and 4.7.1).
Took me like 6 hours to find out it was multidict causing this...
Thanks for the report!
This also happens when multidict is used with nuitka.
@razaqq A temporary workaround is to set the MULTIDICT_NO_EXTENSIONS
environment variable to something before installing via pip, and to then pip install --no-binary multidict multidict
(you might need to add --upgrade --force-reinstall --ignore-installed
too, if the package is already installed).
I encountered this problem, and changed the install lines in my CI/CD pipeline to:
export MULTIDICT_NO_EXTENSIONS=1
pip install --upgrade --force-reinstall --ignore-installed --no-binary :all: <package_dir>
@asvetlov GDB states that the segfault occurs at https://github.com/aio-libs/multidict/blob/master/multidict/_multidict.c#L1595 , though I don't know if that's 100% accurate, as I wasn't working with a debug build (I don't think).
Decref of static type is definitely a mistake but it happens on import failing only as I see.
I wonder why the import multidict
fails at all?
Decref of static type is definitely a mistake but it happens on import failing only as I see.
I wonder why theimport multidict
fails at all?
The fail happens here - https://github.com/aio-libs/multidict/blob/master/multidict/_multidict.c#L1494. And only when using pyinstaller.
@iemelyanov thanks!
I'll have time to work on the fix in 2 hours.
A segfault with 4.7.0 also occurred in open-telemetry/opentelemetry-python#329. This is not using py installer or any other bundling tool.
Can confirm that we've been getting random segfaults with 4.7.1; downgrading to 4.6.1 fixed the issue for us. This is through regular pip installation.
In my case, it crashed with slack-client
2.5.0. Downgrading to 4.6.1 fixed the problem.