Unable to build wheel pyhcl, missing ply.lex
Closed this issue · 8 comments
`mkvirtualenv foo-bar-fudge-tf
New python executable in /home/matthbak/Apps/virtualenvs/foo-bar-fudge-tf/bin/python
Installing setuptools, pip, wheel...done.
virtualenvwrapper.user_scripts creating /home/matthbak/Apps/virtualenvs/foo-bar-fudge-tf/bin/predeactivate
virtualenvwrapper.user_scripts creating /home/matthbak/Apps/virtualenvs/foo-bar-fudge-tf/bin/postdeactivate
virtualenvwrapper.user_scripts creating /home/matthbak/Apps/virtualenvs/foo-bar-fudge-tf/bin/preactivate
virtualenvwrapper.user_scripts creating /home/matthbak/Apps/virtualenvs/foo-bar-fudge-tf/bin/postactivate
virtualenvwrapper.user_scripts creating /home/matthbak/Apps/virtualenvs/foo-bar-fudge-tf/bin/get_env_details
(foo-bar-fudge-tf) (0) matthbak@matthbak-laptop ~/Projects/cpsre/terraform-modules/foo-bar-fudge-tf
$ pip install pyhcl
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7.
Looking in indexes: ...
Collecting pyhcl
Downloading https://files.pythonhosted.org/packages/8c/90/411f698550155532a1375d0367da08c2d0ecc922d4c0362bd4236893238b/pyhcl-0.3.12.tar.gz
Collecting ply<4,>=3.8 (from pyhcl)
Using cached https://files.pythonhosted.org/packages/a3/58/35da89ee790598a0700ea49b2a66594140f44dec458c07e8e3d4979137fc/ply-3.11-py2.py3-none-any.whl
Building wheels for collected packages: pyhcl
Building wheel for pyhcl (setup.py) ... error
Complete output from command /home/matthbak/Apps/virtualenvs/foo-bar-fudge-tf/bin/python -u -c "import setuptools, tokenize;file='/tmp/pip-install-q2Up_m/pyhcl/setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec'))" bdist_wheel -d /tmp/pip-wheel-Pt1ol2 --python-tag cp27:
running bdist_wheel
running build
running build_py
Generating parse table...
Traceback (most recent call last):
File "", line 1, in
File "/tmp/pip-install-q2Up_m/pyhcl/setup.py", line 101, in
"Topic :: Text Processing",
File "/usr/lib/python2.7/distutils/core.py", line 151, in setup
dist.run_commands()
File "/usr/lib/python2.7/distutils/dist.py", line 953, in run_commands
self.run_command(cmd)
File "/usr/lib/python2.7/distutils/dist.py", line 972, in run_command
cmd_obj.run()
File "/home/matthbak/Apps/virtualenvs/foo-bar-fudge-tf/local/lib/python2.7/site-packages/wheel/bdist_wheel.py", line 192, in run
self.run_command('build')
File "/usr/lib/python2.7/distutils/cmd.py", line 326, in run_command
self.distribution.run_command(command)
File "/usr/lib/python2.7/distutils/dist.py", line 972, in run_command
cmd_obj.run()
File "/usr/lib/python2.7/distutils/command/build.py", line 128, in run
self.run_command(cmd_name)
File "/usr/lib/python2.7/distutils/cmd.py", line 326, in run_command
self.distribution.run_command(command)
File "/usr/lib/python2.7/distutils/dist.py", line 972, in run_command
cmd_obj.run()
File "/tmp/pip-install-q2Up_m/pyhcl/setup.py", line 39, in run
self.execute(_pre_install, (), msg="Generating parse table...")
File "/usr/lib/python2.7/distutils/cmd.py", line 349, in execute
util.execute(func, args, msg, dry_run=self.dry_run)
File "/usr/lib/python2.7/distutils/util.py", line 309, in execute
func(*args)
File "/tmp/pip-install-q2Up_m/pyhcl/setup.py", line 31, in _pre_install
import hcl
File "/tmp/pip-install-q2Up_m/pyhcl/src/hcl/init.py", line 1, in
from .api import dumps, load, loads
File "/tmp/pip-install-q2Up_m/pyhcl/src/hcl/api.py", line 2, in
from .parser import HclParser
File "/tmp/pip-install-q2Up_m/pyhcl/src/hcl/parser.py", line 4, in
from .lexer import Lexer
File "/tmp/pip-install-q2Up_m/pyhcl/src/hcl/lexer.py", line 3, in
import ply.lex as lex
ImportError: No module named ply.lex
Failed building wheel for pyhcl
Running setup.py clean for pyhcl
Failed to build pyhcl
Installing collected packages: ply, pyhcl
Running setup.py install for pyhcl ... done
Successfully installed ply-3.11 pyhcl-0.3.12
`
I can repro this with Python 3.7 as well. The problem is that setup.py:39 calls _pre_install which imports hcl package which imports plyin hcl/api.py:2.
Why not just add a generated parsetab.dat file to the repo instead of building one if not exists? That would make this package pip installable.
You could add a new setuptools command so that you could run python setup.py gen_parsetab_dat to generate that file as a prestep before running python setup.py sdist or whatever setup targets depend on that file's existence.
Adding a build step would be fine -- but the parsetab.dat file is a pickle file, and is created by PLY. If you wanted to distribute a sdist/wheel that included it, you would need to ensure that the resulting file is loadable by all versions of python (which I think is possible with certain pickle versions).
I haven't had a need to use it yet, but I wonder if using a pyproject.toml would help here. I believe it has a way to specify the build dependencies so that ply can be installed before it is used.
I think I found a workaround for this issue until it can be solved the proper way. If installed via pip using the github url, it won't produce an error during table generation.
(test-pyhcl) % pip3 freeze
(test-pyhcl) % pip3 install git+https://github.com/virtuald/pyhcl.git@0.3.12#egg=pyhcl
Collecting pyhcl from git+https://github.com/virtuald/pyhcl.git@0.3.12#egg=pyhcl
Cloning https://github.com/virtuald/pyhcl.git (to revision 0.3.12) to /private/var/folders/xw/7w2ry0l91f14wl7vl2ds0xfh0000gn/T/pip-install-k37x8fnn/pyhcl
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing wheel metadata ... done
Collecting ply<4,>=3.8 (from pyhcl)
Using cached https://files.pythonhosted.org/packages/a3/58/35da89ee790598a0700ea49b2a66594140f44dec458c07e8e3d4979137fc/ply-3.11-py2.py3-none-any.whl
Building wheels for collected packages: pyhcl
Building wheel for pyhcl (PEP 517) ... done
Stored in directory: /private/var/folders/xw/7w2ry0l91f14wl7vl2ds0xfh0000gn/T/pip-ephem-wheel-cache-t7thz65d/wheels/70/27/06/77de7cfd11f5e5bd1f905d666c0e0ead0965ee3b5cf165aba4
Successfully built pyhcl
Installing collected packages: ply, pyhcl
Successfully installed ply-3.11 pyhcl-0.3.12
(test-pyhcl) %
watching.
I think I found a workaround for this issue until it can be solved the proper way. If installed via pip using the github url, it won't produce an error during table generation.
(test-pyhcl) % pip3 freeze (test-pyhcl) % pip3 install git+https://github.com/virtuald/pyhcl.git@0.3.12#egg=pyhcl Collecting pyhcl from git+https://github.com/virtuald/pyhcl.git@0.3.12#egg=pyhcl Cloning https://github.com/virtuald/pyhcl.git (to revision 0.3.12) to /private/var/folders/xw/7w2ry0l91f14wl7vl2ds0xfh0000gn/T/pip-install-k37x8fnn/pyhcl Installing build dependencies ... done Getting requirements to build wheel ... done Preparing wheel metadata ... done Collecting ply<4,>=3.8 (from pyhcl) Using cached https://files.pythonhosted.org/packages/a3/58/35da89ee790598a0700ea49b2a66594140f44dec458c07e8e3d4979137fc/ply-3.11-py2.py3-none-any.whl Building wheels for collected packages: pyhcl Building wheel for pyhcl (PEP 517) ... done Stored in directory: /private/var/folders/xw/7w2ry0l91f14wl7vl2ds0xfh0000gn/T/pip-ephem-wheel-cache-t7thz65d/wheels/70/27/06/77de7cfd11f5e5bd1f905d666c0e0ead0965ee3b5cf165aba4 Successfully built pyhcl Installing collected packages: ply, pyhcl Successfully installed ply-3.11 pyhcl-0.3.12 (test-pyhcl) %
Sweet thanks for the workaround. I've added that to my requirements.txt and is working here.
Is there any update on this issue?