Segmentation fault
Closed this issue · 9 comments
The last few days, my CI tests fail with a Segmentation fault (core dumped) error when trying to use django-x509. I noticed that the CI tests of django-x509 fail as well with the same error. I was able to track it down to the latest cryptography release.
Django-x509 requires pyopenssl, which requires cryptography. The latest release of cryptography 2.1.3 was published a few days ago. The changelog contains a single entry:
Updated Windows, macOS, and manylinux1 wheels to be compiled with OpenSSL 1.1.0g.
I was able to work around the segmentation fault by pinning cryptography to 2.1.2. Perhaps django-x509 should also be pinning cryptography, until a proper fix is implemented that allows upgrading cryptography without segfaults.
Hi @dimrozakis,
first of all, thank you for debugging the issue, this news was very helpful. I was going crazy the other day.
I assumed it was an issue with travis-ci since I found out we are not the only ones running into this:
travis-ci/travis-ci#7873
I don't see any report of segmentation faults in the cryptography repo.
To me it sounds absurd that we should change our software because an issue with the CI, I would rather change the CI.
But before deciding what to do, I would like to be sure: this is either a bug in the cryptography library or a bug in travis-ci.
We could resort to pinning cryptography to 2.1.2 temporarily, I would be ok with that, so if you have time and could send a pull request that would be helpful.
Hi @nemesisdesign
I don't think it's a Travis-CI related issue. I got the same error in my Gitlab-CI builds of a larger project that depends on django-x509.
I also repeatedly got the same error when attempting to issue certificates locally, inside a docker container with python 2.7 and django-x509 installed. Downgrading cryptography to 2.1.2 resolved those issues.
I just opened PR #15 to verify that tests indeed pass when pinning cryptography.
Pinning cryptography to 2.1.2 in #15 seems to resolve the issue. This points to the conclusion that it's a cryptography related bug, not a travis ci bug. I'd suggest merging that PR, as a temporary solution, until the actual bug is reported and fixed in cryptography.
What should we do to report the bug upstream? Do you have some more detailed information?
I don't have more detailed information. I reproduced the issue by trying to create a new Cert model and save it. This triggers the keypair and certificate generation, which caused the segmentation fault.
I'd guess that in order to report it, we should come up with a small script that reproduces the issue. We can try to copy the logic of Cert._generate into a standalone function, see which call causes the segfault, and give the script to the devs of cryptography to reproduce and fix the issue. The thing is that django-x509 doesn't directly call cryptography. It calls pyopenssl functions that rely on cryptography under the hood.
I'm too busy currently to work on this though. Feel free to try and reproduce the segfault in a simple script. Otherwise I'll try to do it later today or during the weekend.
@dimrozakis does it happen only on python2 for you or also on python3?
I have one instance of OpenWISP 2, using python2, which I upgraded yesterday and has django-x509 0.3.0 and cryptography 2.1.3 on a debian 7 VM.
On that instance I can create new certificates without any issue.
Maybe is a problem that happens with cryptography 2.1.3 in docker containers with specific images, but I'm just speculating.
I opened an issue here: pyca/cryptography#4015
@nemesisdesign I've only tried this with python2 because the project I'm working on doesn't yet support python3. I'll try to create a dockerfile that reproduces the issue, so that we can track it down to a particular os/python/cryptography/openssl/whatever version.
update: see dockerfile and script to reproduce problem in pyca/cryptography#4015 (comment)
@dimrozakis great work man, thank you very much!
@dimrozakis 0a86ba3 should finally fix this issue... at least I hope so!