pycparser-2.14-py2.py3-none-any.whl causes AssertionError: sorry, but this version only supports 100 named groups
smtakeda opened this issue · 18 comments
Looks like pycparser-2.14-py2.py3-none-any.whl is broken or something misconfigured, while pycparser-2.14.tar.gz is ok. After installing a Snowflake Connector for Python, a connector for Snowflake DB, this error happen:
>pip install -U snowflake-connector-python
>python -c "import snowflake.connector"
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/t/lib/python2.7/site-packages/snowflake/connector/__init__.py", line 21, in <module>
from .connection import SnowflakeConnection
File "/tmp/t/lib/python2.7/site-packages/snowflake/connector/connection.py", line 16, in <module>
from .cursor import SnowflakeCursor
File "/tmp/t/lib/python2.7/site-packages/snowflake/connector/cursor.py", line 30, in <module>
from .file_transfer_agent import (SnowflakeFileTransferAgent)
File "/tmp/t/lib/python2.7/site-packages/snowflake/connector/file_transfer_agent.py", line 29, in <module>
from .s3_util import (SnowflakeS3FileEncryptionMaterial, SnowflakeS3Util,
File "/tmp/t/lib/python2.7/site-packages/snowflake/connector/s3_util.py", line 25, in <module>
from Crypto.Cipher import AES
File "/tmp/t/lib/python2.7/site-packages/Crypto/Cipher/__init__.py", line 78, in <module>
from Crypto.Cipher._mode_ecb import _create_ecb_cipher
File "/tmp/t/lib/python2.7/site-packages/Crypto/Cipher/_mode_ecb.py", line 29, in <module>
from Crypto.Util._raw_api import (load_pycryptodome_raw_lib,
File "/tmp/t/lib/python2.7/site-packages/Crypto/Util/_raw_api.py", line 89, in <module>
Array = ffi.new("char[1]").__class__.__bases__
File "/tmp/t/lib/python2.7/site-packages/cffi/api.py", line 248, in new
cdecl = self._typeof(cdecl)
File "/tmp/t/lib/python2.7/site-packages/cffi/api.py", line 168, in _typeof
result = self._typeof_locked(cdecl)
File "/tmp/t/lib/python2.7/site-packages/cffi/api.py", line 153, in _typeof_locked
type = self._parser.parse_type(cdecl)
File "/tmp/t/lib/python2.7/site-packages/cffi/cparser.py", line 448, in parse_type
return self.parse_type_and_quals(cdecl)[0]
File "/tmp/t/lib/python2.7/site-packages/cffi/cparser.py", line 451, in parse_type_and_quals
ast, macros = self._parse('void __dummy(\n%s\n);' % cdecl)[:2]
File "/tmp/t/lib/python2.7/site-packages/cffi/cparser.py", line 260, in _parse
ast = _get_parser().parse(csource)
File "/tmp/t/lib/python2.7/site-packages/cffi/cparser.py", line 40, in _get_parser
_parser_cache = pycparser.CParser()
File "/tmp/t/lib/python2.7/site-packages/pycparser/c_parser.py", line 87, in __init__
outputdir=taboutputdir)
File "/tmp/t/lib/python2.7/site-packages/pycparser/c_lexer.py", line 66, in build
self.lexer = lex.lex(object=self, **kwargs)
File "/tmp/t/lib/python2.7/site-packages/pycparser/ply/lex.py", line 911, in lex
lexobj.readtab(lextab, ldict)
File "/tmp/t/lib/python2.7/site-packages/pycparser/ply/lex.py", line 233, in readtab
titem.append((re.compile(pat, lextab._lexreflags | re.VERBOSE), _names_to_funcs(func_name, fdict)))
File "/tmp/t/lib/python2.7/re.py", line 194, in compile
return _compile(pattern, flags)
File "/tmp/t/lib/python2.7/re.py", line 249, in _compile
p = sre_compile.compile(pattern, flags)
File "/tmp/t/lib/python2.7/sre_compile.py", line 583, in compile
"sorry, but this version only supports 100 named groups"
AssertionError: sorry, but this version only supports 100 named groups
But if I install it from source pycparser-2.14.tar.gz, it works
>pip install -U ~/Downloads/pycparser-2.14.tar.gz
Processing /Users/stakeda/Downloads/pycparser-2.14.tar.gz
Building wheels for collected packages: pycparser
Running setup.py bdist_wheel for pycparser ... done
Stored in directory: /Users/stakeda/Library/Caches/pip/wheels/6e/f1/96/de2b8478c77d89fe540a5709a70e2cdc4e7331a3543cada3c1
Successfully built pycparser
Installing collected packages: pycparser
Found existing installation: pycparser 2.13
Uninstalling pycparser-2.13:
Successfully uninstalled pycparser-2.13
Successfully installed pycparser-2.14
>python -c "import snowflake.connector"
>
Could you rebuild a new wheel or remove it? Thanks.
Both 2.7.10 and 3.4.3 on Linux (Linux-3.15.1-1.el6.elrepo.x86_64-x86_64-with-centos-6.5-Final) has this issue.
Yup, OpenStack is failing with this new upload too:
Cryptography library is also affected via cffi import.
As a temporary workaround, you can point pip at the Git repo to obtain the sources. The following will point to the commit with the 2.14 tag: git+https://github.com/eliben/pycparser@7f6b3404da3d3184560b04492b64acf5c0ca4ce2#egg=pycparser
Same issue here, took down an auto-scaling cluster of docker containers on AWS, argh...
That github link did not work, but I pointed to release tag and it works fine as:
pip install git+https://github.com/eliben/pycparser@release_v2.14
Another workaround is to add --no-binary pycparser to your requirements file or add said argument to your pip command. That's arguably better, as it will revert to the old behavior (of downloading a .tar.gz from PyPI) and it won't require a Git install.
Odoo and OCB are also impacted by this bug. All our travis builds fail: https://travis-ci.org/OCA/OCB/builds/164508458
Devstack deployment is broken on Murano CI by this issue also https://murano-ci.mirantis.com/jenkins/job/gate-murano-dashboard-ubuntu-glare-nv/333/console
this is a real party
It's funny to see so many CI failing after a new release of a Python component :-) It means that more and more code is well tested and that bugs are spotted more quickly. It's a good thing, right?
Is the plan now to build a new wheel or skip adding a wheel entirely? If adding a new wheel, will you bump the version number of pycparsing or keep it the same?
Asking because my project currently pins pycparser==2.14 and has pip verify the hash. The addition of a wheel (which pip prefers over sdists) caused this setup to break. Just want to know what the plan is going forward so I can accommodate accordingly.
May be useful for someone...Our Devstack also failed because of this bug. We just downgraded pycparser to 2.13 in requirement.txt and disabled 'tempest' service in local.conf. After this, our Devstack installation was successful.
@bmw: I don't have an objection to having wheels, but this experience also teaches me I have to be more careful with changes to pycparser (especially "release" changes like pushing stuff to PyPi). To be honest I haven't yet heard a very strong reason to add a wheel. pycparser is a pure-Python package, so the advantages of having a wheel seem slim to me, and I haven't heard many complaints about it so far.
So I'm inclined to just keep things as they are and not bump anything in the near future.
I'd encourage you to post a wheel on the next release. They are the future of Python packaging. And installing source wheels is slightly faster than a tar.gz source distribution. The difference actually matters at scale. Firefox's CI cumulatively spends several hours a day effectively doing busy work around virtualenv setup because running setup.py (and applying wheels) is non-instantaneous. A few seconds multiplied by hundreds of thousands of test jobs per day matters :)
@indygreg surely you wouldn't rely on package authors to publish wheels then, but have something of your own where you create wheels for projects that don't have them. Otherwise spending so much cycles doesn't sound like a good idea. Or is every project out there except pycparser already publishing wheels?
I can say that OpenStack has been publishing wheels for it's many packages for awhile now.
surely you wouldn't rely on package authors to publish wheels then, but have something of your own where you create wheels for projects that don't have them. Otherwise spending so much cycles doesn't sound like a good idea.
If package authors publish wheels, there's no need for package users to create their own special wheels for their CIs, it reduces duplicated effort for those who want them, and end users also benefit.
Or is every project out there except pycparser already publishing wheels?
There's a big benefit in wheels for Python projects needing to compile C, especially for Windows where setting up a dev environment with a compiler can be a real problem. But for pure Python, source-only isn't generally such a big problem.
Heres's some benefits, and stats on the most-downloaded packages on PyPI (which includes pycparser): 226/360 use wheels. http://pythonwheels.com/
On our CI, pycparser is installed as part of pip install cffi. Different Python versions had different combinations of wheel and source available. Here's how many seconds the combined download and installation operation took.
cffi and pycparser wheels:
0.80
0.92
0.89
0.93
cffi wheel and pycparser source:
2.21
1.97
1.51
cffi source and pycparser wheel:
4.34
4.69
5.76
cffi and pycparser source:
7.30
6.27
5.27
5.61