Spaces in Python path make pip-installed launchers fail on Windows
Closed this issue · 14 comments
Launchers (console_scripts) installed via pip under Windows fail when Python is installed in a path that contains spaces (e.g. C:\Program Files (x86)\..., which will be the default directory for 3.5):
$ pip install vanity
$ vanity
Failed to create process.
It does work, when the package is installed manually via python setup.py install. Also, both ways install the same <packagename>.exe launcher (so this is different from #1997 and #1999).
Via pip install, the shebang in <packagename>-script.py is
#!C:\Program Files (x86)\Python27\python.exe
instead of (via python setup.py install)
#!"C:\Program Files (x86)\Python27\python.exe"
Note that both launcher scripts actually work when executed directly: only the launcher .exe seems to be more strict about quoting. So it may be good to (also) make the launcher more forgiving (is the bug tracker for distlib non-public?).
The error message is different. See issue number 1997 and 1999 (linked above) for the bug related to that SO-thread (which IIUC was was fixed by new .exe-launchers in distlib here).
In this case, the pip seems to write a non-executable shebang line (in contrast to python setup.py install).
Note that this only happens with sdist installs (not with wheels).
As 7.0 switched to building wheels for sdists, one now needs pip install --no-binary <package> <package> to reproduce.
Edit: Want to point out this was run under Windows 10.
I am having the same problem, only it manifests itself because the user name has a space in it.
C:\Users>python -V
Python 3.5.1
C:\Users>pip list
pip (7.1.2)
setuptools (18.2)
C:\Users>pip -V
pip 7.1.2 from c:\users\john hagen\appdata\local\programs\python\python35\lib\site-packages (python 3.5)
C:\Users>pip install cpplint
Collecting cpplint
Using cached cpplint-0.0.6.tar.gz
Installing collected packages: cpplint
Running setup.py install for cpplint
Successfully installed cpplint-0.0.6
C:\Users>where.exe cpplint
C:\Users\John Hagen\AppData\Local\Programs\Python\Python35\Scripts\cpplint.exe
C:\Users>cpplint
failed to create process.
Related stackoverflow posts:
The issue here appears to be the same as one I've experienced with pip 1.5.4 on OS X 10.9.5. It is not a Windows bug per se.
The "shebang" line (hash-bang, or #!) specifying the interpreter to use must not contain a space. The workaround might be to create a symlink which doesn't have a space in the path.
Sadly, the issue is not trivially remediable by mere quoting. See the attached tarball if you want a demo (read the README) or see https://lists.gnu.org/archive/html/bug-bash/2008-05/msg00053.html
Edit: as Janzert indicates below, the problem I reproduce appears not to be related to the Windows problem of the instant issue. This comment pertains to UNIX-like OS. Same symptom with different causes -- leaving this comment above intact to help future readers disambiguate where to be spending time.
Remember Windows itself doesn't do anything with shebang lines. So it is handled by the launcher executable from distlib directly.
A quick skim of the launcher source seems to show that it does explicitly and correctly handle double quotes around the interpreter command.
I have a workaround.
I encountered the same problem in my pandoc-eqnos project. See Issue #6. It occurs on Windows 10 installations, but not 7. I haven't tested 8.
As others have said: A Failed to create process error arises for any console_script when the python path has a space in it. For unknown reasons, pip installs console scripts with unquoted spaced shebang lines on Windows 10. Using python setup.py install on its own has no such problem.
I implemented the workaround in my setup.py. It uses hooks provided by distutils/setuptools to quote shebang lines as needed. Note that there was an additional bug (presumably also arising from pip) that needed to be addressed.
Here is the code for the workaround. You can copy this into your setup.py and enable it by hooking cmdclass into setup().
import ez_setup
ez_setup.use_setuptools()
from setuptools import setup, dist
from setuptools.command.install import install
from setuptools.command.install_scripts import install_scripts
#-----------------------------------------------------------------------------
# Hack to overcome pip/setuptools problem on Win 10. See:
# https://github.com/tomduck/pandoc-eqnos/issues/6
# https://github.com/pypa/pip/issues/2783
# Note that cmdclass must be be hooked into setup().
# Custom install command class for setup()
class custom_install(install):
"""Ensures setuptools uses custom install_scripts."""
def run(self):
super().run()
# Custom install_scripts command class for setup()
class install_scripts_quoted_shebang(install_scripts):
"""Ensure there are quotes around shebang paths with spaces."""
def write_script(self, script_name, contents, mode="t", *ignored):
shebang = str(contents.splitlines()[0])
if shebang.startswith('#!') and ' ' in shebang[2:].strip() \
and '"' not in shebang:
quoted_shebang = '#!"%s"' % shebang[2:].strip()
contents = contents.replace(shebang, quoted_shebang)
super().write_script(script_name, contents, mode, *ignored)
# The custom command classes only need to be used on Windows machines
if os.name == 'nt':
cmdclass = {'install': custom_install,
'install_scripts': install_scripts_quoted_shebang},
# Below is another hack to overcome a separate bug. The
# dist.Distribution.cmdclass dict should not be stored in a length-1 list.
# Save the original method
dist.Distribution._get_command_class = dist.Distribution.get_command_class
# Define a new method that repairs self.cmdclass if needed
def get_command_class(self, command):
"""Pluggable version of get_command_class()"""
try:
# See if the original behaviour works
return dist.Distribution._get_command_class(self, command)
except TypeError:
# If self.cmdclass is the problem, fix it up
if type(self.cmdclass) is tuple and type(self.cmdclass[0]) is dict:
self.cmdclass = self.cmdclass[0]
return dist.Distribution._get_command_class(self, command)
else:
# Something else went wrong
raise
# Hook in the new method
dist.Distribution.get_command_class = get_command_class
else:
cmdclass = {}
Just wanted to confirm @tomduck's findings about Windows 10. The problem I wrote about above was manifest in Windows 10 (I just assumed it affected 7 and 8 as well).
For reference, here is a corresponding (now closed) setuptools issue where this similar problem was reported: pypa/setuptools#398
My own digging found that the problem is not with pip directly, but with setup.py install --single-version-externally-managed: pypa/setuptools#398 (comment)
BUMP!
FYI: The fix for the underlying setuptools issue (pypa/setuptools#398) is now merged there.
I experienced the problem under Win 7. But I have a new workaround (althought I don't understand exactly)
I'm using WinPython 3.4.4.3 which give access to a "WinPython Powershell Prompt.exe".
In this "Powershell", I typed
pip install pyinstaller
pyinstaller.exe : it worked !
But when I start a normal console ("cmd" in the start menu) then I get the
Fatal error in launcher: Unable to create process using '"'
(and I tried to put the folder with space in the PATH, I still get the same error)
So I don't know what the "powershell" does, but it solve the problem.
I guess it's other environment variables...
This appears to be an issue with setuptools and not with pip itself. Closing this as upgrading the version of setuptools seems to be the right answer.