eliben/pycparser

pycparser 2.14 wheel on PyPI differs from source distribution

indygreg opened this issue · 6 comments

indygreg commented

Sometime in the past few hours, fresh virtualenvs using pycparser 2.14 started barfing in CFFI parsing with AssertionError: sorry, but this version only supports 100 named groups. Here is an example stack:

py27 runtests: commands[0] | python setup.py clean develop bdist_wheel
Traceback (most recent call last):
  File "setup.py", line 52, in <module>
    import make_cffi
  File "/home/travis/build/indygreg/python-zstandard/make_cffi.py", line 105, in <module>
    ffi.cdef(source.decode('latin1'))
  File "/home/travis/build/indygreg/python-zstandard/.tox/py27/lib/python2.7/site-packages/cffi/api.py", line 105, in cdef
    self._cdef(csource, override=override, packed=packed)
  File "/home/travis/build/indygreg/python-zstandard/.tox/py27/lib/python2.7/site-packages/cffi/api.py", line 119, in _cdef
    self._parser.parse(csource, override=override, **options)
  File "/home/travis/build/indygreg/python-zstandard/.tox/py27/lib/python2.7/site-packages/cffi/cparser.py", line 299, in parse
    self._internal_parse(csource)
  File "/home/travis/build/indygreg/python-zstandard/.tox/py27/lib/python2.7/site-packages/cffi/cparser.py", line 304, in _internal_parse
    ast, macros, csource = self._parse(csource)
  File "/home/travis/build/indygreg/python-zstandard/.tox/py27/lib/python2.7/site-packages/cffi/cparser.py", line 260, in _parse
    ast = _get_parser().parse(csource)
  File "/home/travis/build/indygreg/python-zstandard/.tox/py27/lib/python2.7/site-packages/cffi/cparser.py", line 40, in _get_parser
    _parser_cache = pycparser.CParser()
  File "/home/travis/build/indygreg/python-zstandard/.tox/py27/lib/python2.7/site-packages/pycparser/c_parser.py", line 87, in __init__
    outputdir=taboutputdir)
  File "/home/travis/build/indygreg/python-zstandard/.tox/py27/lib/python2.7/site-packages/pycparser/c_lexer.py", line 66, in build
    self.lexer = lex.lex(object=self, **kwargs)
  File "/home/travis/build/indygreg/python-zstandard/.tox/py27/lib/python2.7/site-packages/pycparser/ply/lex.py", line 911, in lex
    lexobj.readtab(lextab, ldict)
  File "/home/travis/build/indygreg/python-zstandard/.tox/py27/lib/python2.7/site-packages/pycparser/ply/lex.py", line 233, in readtab
    titem.append((re.compile(pat, lextab._lexreflags | re.VERBOSE), _names_to_funcs(func_name, fdict)))
  File "/home/travis/build/indygreg/python-zstandard/.tox/py27/lib/python2.7/re.py", line 194, in compile
    return _compile(pattern, flags)
  File "/home/travis/build/indygreg/python-zstandard/.tox/py27/lib/python2.7/re.py", line 249, in _compile
    p = sre_compile.compile(pattern, flags)
  File "/home/travis/build/indygreg/python-zstandard/.tox/py27/lib/python2.7/sre_compile.py", line 583, in compile
    "sorry, but this version only supports 100 named groups"
AssertionError: sorry, but this version only supports 100 named groups

This CI environment is pinning versions, so it should be deterministic.

Strangely, I was able to reproduce this error on a fresh virtualenv but not an existing one. I diffed the contents of the virtualenvs and saw that a number of .py files related to pycparser had changed!

Looking at https://pypi.python.org/pypi/pycparser, there is a pycparser-2.14.tar.gz uploaded 2015-06-10. There is also a pycparser-2.14-py2.py3-none-any.whl uploaded today.

If you compare the source in the .tar.gz from the wheel, it varies.

It looks like the wheel just uploaded to PyPI was cut from a different commit than what the tar.gz was cut from. And since modern versions of pip prefer downloading wheels over the source distribution, pip is downloading the wheel.

Please delete the bad wheel from PyPI. Please also consider cutting a 2.15 release that is the same commit as 2.14 but with matching sources.

smtakeda commented

came across the same issue. reported in #147

indygreg commented

This is a dupe of #147 and there are workarounds listed in that issue until the bad wheel is yanked from PyPI.

  • 👍1
rodrigc commented

Saw the same problem building https://github.com/buildbot/buildbot

@tardyp the buggy pycparser wheel is affecting latest builds of buildbot

simquad commented

This is also affecting the install of Sentry

eliben commented

I removed the wheel [it was uploaded following a request in #100]

  • 👍11