TexteaInc/funix

pip install writes to system path with `-e`

forrestbao opened this issue · 12 comments

forrestbao commented

It is really weird that pip detects to switch to user installation (first line printed out below) because I did not sudo. But then the error says that it wants to write to /usr/lib.

I could do python3 -m pip install --user . but not $ python3 -m pip install --user -e . either.

I executed the command below from the root directory a cloned copy of PyDataFront.

$ python3 -m pip install -e .                                                                         
Defaulting to user installation because normal site-packages is not writeable
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Obtaining file:///home/forrest/Textea/PyDataFront
  Installing build dependencies ... done
  Checking if build backend supports build_editable ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Installing collected packages: pydatafront
  Attempting uninstall: pydatafront
    Found existing installation: pydatafront 0.0.0
    Uninstalling pydatafront-0.0.0:
      Successfully uninstalled pydatafront-0.0.0
  Running setup.py develop for pydatafront
    error: subprocess-exited-with-error
    
    × python setup.py develop did not run successfully.
    │ exit code: 1
    ╰─> [32 lines of output]
        running develop
        /usr/lib/python3/dist-packages/setuptools/command/easy_install.py:158: EasyInstallDeprecationWarning: easy_install command is deprecated. Use build and pip and other standards-based tools.
          warnings.warn(
        WARNING: The user site-packages directory is disabled.
        /usr/lib/python3/dist-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
          warnings.warn(
        error: can't create or remove files in install directory
        
        The following error occurred while trying to add or remove files in the
        installation directory:
        
            [Errno 13] Permission denied: '/usr/local/lib/python3.10/dist-packages/test-easy-install-769272.write-test'
        
        The installation directory you specified (via --install-dir, --prefix, or
        the distutils default setting) was:
        
            /usr/local/lib/python3.10/dist-packages/
        
        Perhaps your account does not have write access to this directory?  If the
        installation directory is a system-owned directory, you may need to sign in
        as the administrator or "root" account.  If you do not have administrative
        access to this machine, you may wish to choose a different installation
        directory, preferably one that is listed in your PYTHONPATH environment
        variable.
        
        For information on other options, you may wish to consult the
        documentation at:
        
          https://setuptools.pypa.io/en/latest/deprecated/easy_install.html
        
        Please make the appropriate changes for your system and try again.
        
        [end of output]
    
    note: This error originates from a subprocess, and is likely not a problem with pip.
  WARNING: No metadata found in /home/forrest/.local/lib/python3.10/site-packages
  Rolling back uninstall of pydatafront                                                                                                           
  Moving to /home/forrest/.local/lib/python3.10/site-packages/pydatafront-0.0.0.dist-info/
   from /home/forrest/.local/lib/python3.10/site-packages/~ydatafront-0.0.0.dist-info
error: subprocess-exited-with-error

× python setup.py develop did not run successfully.
│ exit code: 1
╰─> [32 lines of output]
    running develop
    /usr/lib/python3/dist-packages/setuptools/command/easy_install.py:158: EasyInstallDeprecationWarning: easy_install command is deprecated. Use build and pip and other standards-based tools.
      warnings.warn(
    WARNING: The user site-packages directory is disabled.
    /usr/lib/python3/dist-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
      warnings.warn(
    error: can't create or remove files in install directory
    
    The following error occurred while trying to add or remove files in the
    installation directory:
    
        [Errno 13] Permission denied: '/usr/local/lib/python3.10/dist-packages/test-easy-install-769272.write-test'
    
    The installation directory you specified (via --install-dir, --prefix, or
    the distutils default setting) was:
    
        /usr/local/lib/python3.10/dist-packages/
    
    Perhaps your account does not have write access to this directory?  If the
    installation directory is a system-owned directory, you may need to sign in
    as the administrator or "root" account.  If you do not have administrative
    access to this machine, you may wish to choose a different installation
    directory, preferably one that is listed in your PYTHONPATH environment
    variable.
    
    For information on other options, you may wish to consult the
    documentation at:
    
      https://setuptools.pypa.io/en/latest/deprecated/easy_install.html
    
    Please make the appropriate changes for your system and try again.
    
    [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
TURX commented

You might have forgotten to activate your venv?

forrestbao commented

This should not be related to venv. The first line in the printout shows that pip already knows to install to a user directory rather than a system one.

It seems to be this problem: pypa/pip#7953

TURX commented

I have no issue installing and running it with -e on Docker image python:3.10-bullseye.

root@2b83b188f1de:/app# pip uninstall pydatafront
Found existing installation: pydatafront 0.0.1
Uninstalling pydatafront-0.0.1:
  Would remove:
    /root/.local/lib/python3.10/site-packages/pydatafront-0.0.1.dist-info/*
    /root/.local/lib/python3.10/site-packages/pydatafront/*
Proceed (Y/n)? y
  Successfully uninstalled pydatafront-0.0.1
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
root@2b83b188f1de:/app# python3 -m pip install --user -e .
Obtaining file:///app
  Installing build dependencies ... done
  Checking if build backend supports build_editable ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: flask in /root/.local/lib/python3.10/site-packages (from pydatafront==0.0.1) (2.1.2)
Requirement already satisfied: flask-cors in /root/.local/lib/python3.10/site-packages (from pydatafront==0.0.1) (3.0.10)
Requirement already satisfied: Jinja2>=3.0 in /root/.local/lib/python3.10/site-packages (from flask->pydatafront==0.0.1) (3.1.2)
Requirement already satisfied: itsdangerous>=2.0 in /root/.local/lib/python3.10/site-packages (from flask->pydatafront==0.0.1) (2.1.2)
Requirement already satisfied: Werkzeug>=2.0 in /root/.local/lib/python3.10/site-packages (from flask->pydatafront==0.0.1) (2.1.2)
Requirement already satisfied: click>=8.0 in /root/.local/lib/python3.10/site-packages (from flask->pydatafront==0.0.1) (8.1.3)
Requirement already satisfied: Six in /root/.local/lib/python3.10/site-packages (from flask-cors->pydatafront==0.0.1) (1.16.0)
Requirement already satisfied: MarkupSafe>=2.0 in /root/.local/lib/python3.10/site-packages (from Jinja2>=3.0->flask->pydatafront==0.0.1) (2.1.1)
Installing collected packages: pydatafront
  Running setup.py develop for pydatafront
Successfully installed pydatafront-0.0.1
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
WARNING: You are using pip version 22.0.4; however, version 22.1.2 is available.
You should consider upgrading via the '/usr/local/bin/python3 -m pip install --upgrade pip' command.
root@2b83b188f1de:/app# python3 -m pydatafront
usage: __main__.py [-h] [--host HOST] [--port PORT] this_class main_class
__main__.py: error: the following arguments are required: main_class
forrestbao commented

This is because you are root.

TURX commented

I have it installed in ~/.local.

root@2b83b188f1de:~/.local/lib/python3.10/site-packages# ls | grep pydatafront
pydatafront.egg-link
forrestbao commented

Still, your ~ is /root/

TURX commented

Let me test in my school lab.

TURX commented

I have installed the current main branch on my home dir in my school lab where I have no privileged access.

[ruixuan@rockhopper-01] (32)$ python3 -m pip install --user -e .
Obtaining file:///afs/cs.wisc.edu/u/r/u/ruixuan/test/PyDataFront
  Installing build dependencies ... done
  Checking if build backend supports build_editable ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: flask in /usr/lib/python3/dist-packages (from pydatafront==0.0.1) (1.1.1)
Collecting flask-cors
  Downloading Flask_Cors-3.0.10-py2.py3-none-any.whl (14 kB)
Requirement already satisfied: Six in /usr/lib/python3/dist-packages (from flask-cors->pydatafront==0.0.1) (1.14.0)
Installing collected packages: flask-cors, pydatafront
  Running setup.py develop for pydatafront
Successfully installed flask-cors-3.0.10 pydatafront
[ruixuan@rockhopper-01] (33)$ ls
backend/  examples/  frontend/  main.py  pyproject.toml  README.md  setup.cfg
[ruixuan@rockhopper-01] (34)$ python3 -m pydatafront
usage: __main__.py [-h] [--host HOST] [--port PORT] this_class main_class
__main__.py: error: the following arguments are required: main_class
[ruixuan@rockhopper-01] (35)$ python3 -m pip uninstall pydatafront
Found existing installation: pydatafront 0.0.1
Uninstalling pydatafront-0.0.1:
  Would remove:
    /afs/cs.wisc.edu/u/r/u/ruixuan/.local/lib/python3.8/site-packages/pydatafront.egg-link
Proceed (Y/n)? y
  Successfully uninstalled pydatafront-0.0.1
TURX commented

You might try upgrading your pip first? python3 -m pip install --upgrade pip

forrestbao commented

I tried to upgrade. No clue. I can setPYTHONPATH for now as a temporarily workaround.

forrestbao commented

I just tried on a freshly install Lubuntu 22.04 box. Same problem. Pip is already upgraded.

$ python3 -m pip install --upgrade pip
Defaulting to user installation because normal site-packages is not writeable
Requirement already satisfied: pip in /usr/lib/python3/dist-packages (22.0.2)
Collecting pip
  Downloading pip-22.2.2-py3-none-any.whl (2.0 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.0/2.0 MB 3.3 MB/s eta 0:00:00
Installing collected packages: pip
  WARNING: The scripts pip, pip3 and pip3.10 are installed in '/home/forrest/.local/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.  
Successfully installed pip-22.2.2

It's really weird that pip insists to write to /usr/local/lib/python3.10/dist-packages/ instead of my home directory.

$ python3 -m pip install  --user -e .
Obtaining file:///home/forrest/repos/PyDataFront
  Installing build dependencies ... done
  Checking if build backend supports build_editable ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Installing collected packages: pydatafront
  Running setup.py develop for pydatafront
    error: subprocess-exited-with-error
    
    × python setup.py develop did not run successfully.
    │ exit code: 1
    ╰─> [32 lines of output]
        running develop
        /usr/lib/python3/dist-packages/setuptools/command/easy_install.py:158: EasyInstallDeprecationWarning: easy_install command is deprecated. Use build and pip and other standards-based tools.
          warnings.warn(
        WARNING: The user site-packages directory is disabled.
        /usr/lib/python3/dist-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
          warnings.warn(
        error: can't create or remove files in install directory
        
        The following error occurred while trying to add or remove files in the
        installation directory:
        
            [Errno 13] Permission denied: '/usr/local/lib/python3.10/dist-packages/test-easy-install-5961.write-test'
        
        The installation directory you specified (via --install-dir, --prefix, or
        the distutils default setting) was:
        
            /usr/local/lib/python3.10/dist-packages/
        
        Perhaps your account does not have write access to this directory?  If the
        installation directory is a system-owned directory, you may need to sign in
        as the administrator or "root" account.  If you do not have administrative
        access to this machine, you may wish to choose a different installation
        directory, preferably one that is listed in your PYTHONPATH environment
        variable.
        
        For information on other options, you may wish to consult the
        documentation at:
        
          https://setuptools.pypa.io/en/latest/deprecated/easy_install.html
        
        Please make the appropriate changes for your system and try again.
        
        [end of output]
    
    note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× python setup.py develop did not run successfully.
│ exit code: 1
╰─> [32 lines of output]
    running develop
    /usr/lib/python3/dist-packages/setuptools/command/easy_install.py:158: EasyInstallDeprecationWarning: easy_install command is deprecated. Use build and pip and other standards-based tools.
      warnings.warn(
    WARNING: The user site-packages directory is disabled.
    /usr/lib/python3/dist-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
      warnings.warn(
    error: can't create or remove files in install directory
    
    The following error occurred while trying to add or remove files in the
    installation directory:
    
        [Errno 13] Permission denied: '/usr/local/lib/python3.10/dist-packages/test-easy-install-5961.write-test'
    
    The installation directory you specified (via --install-dir, --prefix, or
    the distutils default setting) was:
    
        /usr/local/lib/python3.10/dist-packages/
    
    Perhaps your account does not have write access to this directory?  If the
    installation directory is a system-owned directory, you may need to sign in
    as the administrator or "root" account.  If you do not have administrative
    access to this machine, you may wish to choose a different installation
    directory, preferably one that is listed in your PYTHONPATH environment
    variable.
    
    For information on other options, you may wish to consult the
    documentation at:
    
      https://setuptools.pypa.io/en/latest/deprecated/easy_install.html
    
    Please make the appropriate changes for your system and try again.
    
    [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
forrestbao commented

Solution found here #24 (comment)