Python is deprecating `distutils.cfg`

If you have been directed here from a pip warning, you don't need to take any action at this stage. Nothing is currently expected to break until Python 3.12, and a full solution will be implemented well before then.

Note that as of 3.10.2, this issue no longer affects Python 3.10 and later.

Quoting @uranusjr from Homebrew/discussions#1405:

I am one of the maintainers of pip, the Python package installer. We are recently informed that Homebrew is currently using distutils.cfg to configure package installation schemes, and we believe it is best for Homebrew to remove the usage.

Python is currently deprecating distutils, and will remove the module entirely in 3.10 (PEP 632). As a part of the transition, pip is moving all its current distutils usages to the sysconfig module instead, and a user recently reported pip would no longer work correctly with Homebrew-compiled Python after the transision. This is due to Homebrew’s usage of distutils.cfg, which is no longer supported in sysconfig.

The distutils.cfg file (and its various variants) has long been disfavoured by Python core maintainers, and the recommendation, if redistributors wish to provide custom install schemes, has been to directly patch distutils and sysconfig instead. This is the current approach taken by all Linux distributions that we have contact with, and the community is working toward providing better support to the patching process (b.p.o-43976). I’m recommending Homebrew to also migrate away from the distutils.cfg usage, since it is going to break while the community continuously migrate away from distutils, and stop working entirely when Python 3.12 lands, and join the conversation with other Python redistributors to improve sysconfig to support Homebrew’s needs.

Reposting here so it won't get buried in the Discussions page. We probably want to fix python@3.9 so it can be easily re-used for python@3.10.

has been to directly patch distutils and sysconfig instead

This links a bit with the pth situation, where it might also be benficial to patch to prevent pip uninstalling brew packages.

the community is working toward providing better support to the patching process (b.p.o-43976)

Perhaps best to wait for the outcome of this before acting.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

As a related follow-up, a conversation happend during PyCon US last week between PyPA and Linux distribution folks around how the make distribution package managers (e.g. brew) and Python package managers (e.g. pip) work well together. We are working on an informational PEP to propose some new mechanisms and best practices, which may interest Homebrew maintainers. We plan to continue the conversaion on the linux-sig mailing list (not started yet) so it would be awesome if Homebrew maintainers could follow it as well (the list is called linux-sig for historical reasons, but the topic applies to many downstream Python distributors including Hombrew, MacPorts, Anaconda, etc.)

Related: geofft/peps#3

Hi @uranusjr, thanks for that info. I don't know about anyone else on the Homebrew team but I've joined the linux-sig list you linked for now. Reading through that draft PEP now (thanks Carlo).

I was sent here by a deprecation notice from pip. It's not quite clear what I should do. Just wait until a solution has been figured out?

Just wait until a solution has been figured out?

For users: yes. AFAIK nothing is broken yet.

The warning still seems a bit early considering a proper solution forward has not yet been agreed in the Python community. The discussion in the linked draft PEP appears to have stalled for now.

It seems the pip team might be trying to push us into applying custom patches. This is not a long term solution at all and is not something we want to maintain in the long run.

Assuming sysconfig is what pip are wanting to use going forward, we actually already do "fix" some sys values via a sitecustomize.py file so I can look into seeing if it works for sysconfig perhaps, but I would like us to focus on getting a better solution integrated into Python 3.10 (or probably 3.11 at this point) as this is becoming a cat and mouse game with different tools using different things - it's not universally sysconfig sadly, even when distutils vanishes.

It seems the pip team might be trying to push us into applying custom patches. This is not a long term solution at all and is not something we want to maintain in the long run.

IYAM it's not something we want to maintain in the short term either and if upstream prefers to ship a broken system, Homebrew's stance on patches makes it perfectly fine for us to ship said broken version.

I think the main problem is I failed to communicate the situation clearly, so please let me try to describe what we’re facing and hopefully provide enough context for the situation.

Python 3.12 is going to remove distutils, and along with it, support for distutils.cfg. This means that how Homebrew is currently distributing Python will break, and we need to do something to avoid Homebrew being unable to distribute a working Python 3.12 when it lands. The approach currently persumed by the Linux community could be a good way to resolve this, but they do not have the same hard deadline on 3.12 like Homebrew (although it is preferred if they could achieve this at that time), because they already have patches to Python that would work for both current Python versions and past 3.12. Homebrew therefore would need to either more actively drive that discussion forward, or persume a downstream patch solution like Linux distributions currently do to keep things working.

Pip, like Homebrew, is currently relying on distutils functionalities, and have the same 3.12 deadline. We are therefore trying to rewrite things in pip to keep it working on Python 3.12 onwards. However, some of the new code, since it can’t depend on distutils and distutils.cfg, is incompatible with how Homebrew distributes Python. This is problematic since Homebrew allows users to freely upgrade their pip version in a Hombrew-distributed Python, and once pip completely migrates off the current distutils implementation (sometime before 3.12 lands), those users upgrading to a new pip version will see pip break on their Hombrew-distributed Python, and pip is going to face an impossible problem—we won’t be able to support a Homebrew-distributed Python because it uses mechanisms pip can’t use, but also can’t stop people from upgrading pip agaist those Python installations.

This is what the warning is about: to raise awareness to the issue, and hopefully gather enough resources and contribute to a resolution. We want pip to work on both Python 3.12 (Homebrew or otherwise), and a Python (<3.12) distributed by Homebrew, but this is not possible without help from Homebrew’s Python distribution maintainers. (Note that pip developers are not CPython developers, so we are not in the best position to come up with a solution that suits Homebrew best, although we are definitely good at checking whether a proposed solution would work!)

but this is not possible without help from Homebrew’s Python distribution maintainers.

That's gonna be really hard because Homebrew doesn't have a concept of package maintainers. All Homebrew maintainers maintain all packages (or the brew tool) and the public can modify packages however they see fit, as long as it's within the rules defined in docs.brew.sh

Then I guess that sort of enhance my belief we should keep the warning until we can find someone in the public willing to come up with something :)

Please do also provide any suggestions how we could improve the warning’s presentation to maximise the effect as well!

Hello,

I am redirected here due to the deprecation warning when I install packages using pip3 (for python3.9.6). I didn't see any break in my projects and all the packages I installed so far work well. What should we do know?

Thank you very much.

Would the Allow Python distributors to add custom site install schemes feature make it easy enough for Homebrew (& others) to switch from distutils to sysconfig?

If not, what can the Python / sysconfig people do to make Homebrew's (& others') transitions easier?

If so, wouldn't it make sense for the Python / sysconfig maintainers to implement that feature before removing distutils?

What should we do know?

Nothing for now since pip is not going to break anything for existing Python versions. But you probably want to come back to this issue before Homebrew tries to upgrade Python to 3.11 (or more critically 3.12) and see if the issue is resolved yet and decide if you want to apply the upgrade.

If so, wouldn't it make sense for the Python / sysconfig maintainers to implement that feature before removing distutils?

Of course it does, but in the end it’s up to the downstream distributors to tell CPython what they want. If no-one says anything, CPython maintainers can only assume everyone is fine with the removal. (Which is what the b.p.o. issue you linked is expressing; FFY00 is the Python distributor for Arch Linux IIRC, so CPython maintainers are trying to understand their needs and adjust sysconfig accordingly.)

it’s up to the downstream distributors to tell CPython what they want

Has anyone from Homebrew told CPython what they want / need?

If not, why?

If so, will CPython provide it?

If they won't provide it, why?

If they will provide it, what is left to discuss? (I assume this isn't the scenario, since discussion would presumably have ended if it were so)

Has anyone from Homebrew told CPython what they want / need?

Currently we provide a distutils.cfg as such:

[install]
prefix=/usr/local
[build_ext]
include_dirs=/usr/local/include:/usr/local/opt/openssl@1.1/include:/usr/local/opt/sqlite/include
library_dirs=/usr/local/lib:/usr/local/opt/openssl@1.1/lib:/usr/local/opt/sqlite/lib

To my understanding, the way to configure build_ext hasn't changed yet(?), though I've not been closely following setuptools changes. For pip, the discussion is mostly about the install config items, which can now be done by configuring sysconfig (even for things still using distutils once Python 3.10 releases).

Has anyone from Homebrew told CPython what they want / need?

It's no different than bpo-43976, so there's not much to add there. Having a site-packages that lies in a different prefix to the stdlib prefix isn't a unique situation.

I'm happy however to be involved in any necessary discussions however.

what is left to discuss?

Honestly, I don't know. I can leave a comment as a "poke" but maybe that's more annoying than helpful.

It's no different than bpo-43976, so there's not much to add there. Having a site-packages that lies in a different prefix to the stdlib prefix isn't a unique situation.

bpo-43976 seems to indicate that Homebrew will have to implement a patch for the Python install. You (@Bo98) & @SMillerDev indicated that you don't want to do that for the long term, and possibly not even for the short term. Is that correct?

@uranusjr If a patch would be required, would it be just some simple boilerplate around a few settings, rather than a significant effort? If so, would the boilerplate be stable for a long time, or might it need to be frequently changed for new Python versions?

If the patch would be fairly simple & stable, would the Homebrew maintainers be amenable to using such a patch? If you aren't amenable to using a patch that is both simple & stable, why?

If the patch should be fairly simple, could someone from CPython / Python / pip / whatever provide a simple patch with placeholders for the values that Homebrew (or others) would supply? Could they commit to providing updates for the simple patch of it must be modified for a new Python version?

If a simple patch would cover most downstream use cases, could CPython (or whoever) just provide the equivalent of that patch either as install options for the values themselves and/or as an install option to specify a config file that provides the values?

Sorry if this has all been answered before, but there are too many comments to quickly find the current status.

Can the current status of everything be inserted into the initial issue comment, so everyone can get up to speed quickly?

Thanks for any info.

bpo-43976 seems to indicate that Homebrew will have to implement a patch for the Python install.

No, bpo-43976 is a request to add a system to customise without applying patches. We don't consider adding new files as a patch (we already do this with sitecustomize.py and distutils.cfg).

Can the current status of everything be inserted into the initial issue comment, so everyone can get up to speed quickly?

I've added a short note for end users seeing the pip warning, since that'll be what most of the traffic here will be. People interested in the implementation details can follow along with the comments.

@Bo98 I am not faimilar with the situation for setuptools either, but setuptools is known for keeping compatibility above almost everything, so I wouldn’t be surprised if they keep supporting distutils.cfg until the end of time. But you are likely still going to need to change something, because distutils.cfg should currently be placed in {python_prefix}/lib/{python_version}/distutils, which is going to disappear entirely in 3.12, so at the very least that file needs to go somewhere else.

@rgoldberg See Debian’s patch. Essentially you need make get_default_scheme return something else than the default scheme (which would be under the default prefix e.g. /usr/local/opt/python@3.10). I’m not sure if Homebrew needs to be able to install Python packages to that default prefix; if it doesn’t, maybe Homebrew can simply patch out posix_prefix to point to /usr/local (but remember to account for virtual environments; which can be detected by checking sys.prefix and sys.base_prefix). If the ability to install into the default prefix must be kept like Debian (their Python’s default prefix is /usr, which is also the prefix for e.g. apt install python3-pip, but they want pip to use /usr/local as prefix), then you need one additional scheme and some mechanism to switch between them (Debian uses an environment variable).

I think this also somewhat answers your other question about why there is not a commonly maintained patch, or even upstreamed back into CPython. Each downstream distributor does things a bit differently, so their patches also need to slightly differ, and they can’t agree on what to upstream in the first place (the metioned in-draft PEP is an effort toward some kind of common ground, but it’s just very hard). I believe Red Hat’s patch is actually more popular than Debian’s (also used by at least Cygwin and MSYS2), but their setups are also significantly different from Homebrew’s. Homebrew’s setup is (from my very limited experience) actually most similar to NixOS(?) but Nix does some very fancy stuff because they have very ambitious goals not perused by anyone else.

bpo-43976 seems to indicate that Homebrew will have to implement a patch for the Python install.

No, bpo-43976 is a request to add a system to customise without applying patches. We don't consider adding new files as a patch (we already do this with sitecustomize.py and distutils.cfg).

Sorry. I only scanned the issues & its comments. I saw some talk in the comments about "patching" sysconfig.py, so I assumed that people were suggesting patching it instead of providing a proper config file.

See Debian’s patch

@uranusjr must Debian's patch be updated frequently for new Python versions, or is it fairly stable?

If Python requires patching instead of supporting a config file, if a patch similar to Debian's will work for Homebrew & would be stable for many Python versions, 3 dozen or so line changes in a single file to get Python working in Homebrew seems preferable to providing it broken, and to not providing new versions at all. It might not be as ideal as Python supporting a config file, but it's better than the alternatives, if Python forces patching upon everyone.

From what I hear, the patch is pretty stable.

Does anyone know when this issue will be solved?

No. Someone still needs to step up to contribute a fix for this (which includes figuring out what that fix should be).

Nothing is expected to break before python@3.12, but the deprecation error is showing up much earlier in python@3.8 via Homebrew. Has there been any movement with bpo-43976?

I really appreciate those that dedicate time to Python and the PyPA. Likewise, Homebrew. Thanks for all you do. I'm coming from the end-user perspective and thinking do I need to see this big warning from pip for 1-2 years? I'm glad you maintainers are working together and ensuring the future is smooth!

What is the appropriate method to specifically silence this particular depreciation warning, especially as many of us would probably not be involved in contributing any fixes to address this issue?

how to resole this problem?

I greatly appreciate the work all of you dedicated to maintaining Python packages for Homebrew are doing, but do I really need to see this depreciation warning multiple times, or at all, when I run pip? Especially if you consider the fact that the link in the warning message leads to this page, which begins with "If you have been directed here from a pip warning, you don't need to take any action at this stage."

To be frank, this is just ridiculous:

The key '--no-python-version-warning' doesn't do anything.

Okay I've found where it is. To remove these messages, first, "pin" your python so it won't update:
brew pin python@3.9 (replace python@3.9 with the package name of your Python)

Considering pip, AFAIK it won't update unless you explicitly request the update, so there's no need to figure out a way to "pin" it. Just don't update it manually.

Second, open this file:
/usr/local/lib/python3.9/site-packages/pip/_internal/locations/__init__.py (replace python3.9 with the appropriate version)

Third, find and remove these lines:

        deprecated(
            "Configuring installation scheme with distutils config files "
            "is deprecated and will no longer work in the near future. If you "
            "are using a Homebrew or Linuxbrew Python, please see discussion "
            "at https://github.com/Homebrew/homebrew-core/issues/76621",
            replacement=None,
            gone_in=None,
        )

Ta-da! Now the annoying messages will go away.

brew pin python@3.9

FYI: Some things are going to complain about this, even if they don't actually need a newer python. Ran into this today when trying to install icdiff:

Error: You must `brew unpin python@3.9` as installing icdiff requires the latest version of pinned dependencies

I reinstalled python3 from scratch in my MacBook:
brew reinstall python@3.9

Then:

pip3 install numpy
pip3 install opencv-contrib-python

It worked.

Hi, let me echo the thanks to the homebrew contributors for this amazing ecosystem which I use literally every day.

However, can we please remove this end-user warning? The vast majority of homebrew users are not homebrew developers (perhaps unfortunately) and this clutter is actually damaging because it makes it harder to spot actionable error messages from pip (e.g., from package conflicts).

Alternately, perhaps we [someone with better knowledge of how to make homebrew contributions] could add an environmental variable that would silence the warning (like BASH_SILENCE_DEPRECATION_WARNING on macOS)?

However, can we please remove this end-user warning?

You can remove it by fixing the problem in python. We generally don't hide warnings that upstream throws because they have a reason to be there

You can remove it by fixing the problem in python.

(Clarification to avoid misunderstanding) This means Homebrew’s Python distribution, not the actual upstream Python source code.

No, it means the actual Python upstream project since that's where the warnings come from.

But the upstream Python will not fix this. distutils.cfg and the entire distutils module is being deprecated for removal, the only way to “fix” this in upstream Python is to actually remove the module, (which will happen two versions later, in 3.12) and break Homebrew’s Python entirely.

I think we are confused by two meanings of "fix" here. The first is "get rid of the distracting warning message"; this could be fixed today, either within the upstream pip or in a homebrew-specific patch. The second is "fix the underlying problem by removing the homebrew dependence on distutils".

The former fix would be aesthetically pleasing, since I believe that the warning message actually serves little to no purpose and is probably at least mildly harmful. But since it actually came from the pip developers I guess they disagree, or at least want to absolutely ensure that the issue remains on peoples' minds. I guess this thread is evidence that this aspect is actually working?....

The latter fix is going to happen eventually, I suppose, but it hasn't made it to the front of the queue.

The first is "get rid of the distracting warning message"; this could be fixed today, either within the upstream pip or in a homebrew-specific patch

Homebrew does not agree with making Homebrew-specific patches whenever avoidable. In this case: it's easily avoidable.

The latter fix is going to happen eventually, I suppose, but it hasn't made it to the front of the queue.

It'll happen when someone in the Homebrew community does the necessary work or a new version of Python is released which we don't include in Homebrew until this problem is fixed.

For reference, the code for the deprecation warning you're seeing lives here: https://github.com/pypa/pip/blob/e0c7f24ee4fd9d3279b25acfd978c080a212b5c1/src/pip/_internal/locations/__init__.py#L351-L357 Thanks @y4rr for tracking that down.

You may wish to consider their suggested solution to this. (Except the brew pin part -- that's not something I would personally recommend doing.)

The first is "get rid of the distracting warning message"; this could be fixed today, either within the upstream pip or in a homebrew-specific patch

Homebrew does not agree with making Homebrew-specific patches whenever avoidable. In this case: it's easily avoidable.

Homebrew does not agree with who making Homebrew-specific patches? In this case, who can avoid it?

It'll happen when someone in the Homebrew community does the necessary work or a new version of Python is released which we don't include in Homebrew until this problem is fixed.

I predict that Homebrew will continue to ship new Python versions regardless of whether CPython and PyPA follow Homebrew’s aesthetic taste in patching. (To keep myself honest, I'm setting a reminder for 2023 to check in on this thread.)

Homebrew does not agree with who making Homebrew-specific patches? In this case, who can avoid it?

Homebrew wants patches submitted to upstream projects and then only applied here when they are accepted upstream. Sometimes this is not avoidable due to how Homebrew is installed. In this case: it is avoidable so we would rather not have a patch.

I predict that Homebrew will continue to ship new Python versions regardless of whether CPython and PyPA follow Homebrew’s aesthetic taste in patching. (To keep myself honest, I'm setting a reminder for 2023 to check in on this thread.)

We'll ship them until this warning turns into a breaking error at which point:

someone (quite probably not a maintainer) will fix the problem
Homebrew will not release new Python versions

It seems unlikely to me that Homebrew will ultimately abandon packaging Python over the aesthetics of patching it, when every other distro manages to ship Python.

In any case, I've filed pypa/pip#10712 so now upstream formally knows about the issue. It's pretty clearly their bug.

Note also that the warning explicitly mentions this issue. This means that whenever anyone posts pip output from a homebrew distribution, it points here, even when it is not relevant.

Actually there is a lingering problem for all of us currently using python@3.10 already, as i just described over in Homebrew/brew#12555 in a lot more detail:

If you're using python@3.10 and install libs with pip3(.10) their scripts (e.g., jupyter, virtualenv, pip) end up in the wrong place (/usr/local/Cellar/python@3.10/3.10.1/Frameworks/Python.framework/Versions/3.10/bin) that no one told you to put in your PATH and that's sadly specific to 3.10.1 (not whole 3.10 as you're used to). You're likely still using old scripts from /usr/local/bin put there by pip3.9. Currently there is no >= 3.10 stable scripts PATH, which this feature request suggests to change.

The plan is still working with upstream to get this fixed in Python 3.11. This is involving co-operation with other packagers as well (Debian are for example checking whether the solution proposed suits their case).

This isn't a situation unique to Homebrew. Fedora for example has incorporated a patch to override the change in pip 21.3 to use sysconfig (and we will do the same until the full solution is finalised). Many others have simply not shipped pip 21.3 yet.

Hi, are there any workarounds to this issue? It seems to be breaking my module when I try to install from pip and brew.

Python: 3.9
Homebrew: 3.3.12

pip3 install highdicom
DEPRECATION: Configuring installation scheme with distutils config files is deprecated and will no longer work in the near future. If you are using a Homebrew or Linuxbrew Python, please see discussion at https://github.com/Homebrew/homebrew-core/issues/76621
Requirement already satisfied: highdicom in /opt/homebrew/lib/python3.9/site-packages (0.14.0)
Requirement already satisfied: pylibjpeg>=1.3 in /opt/homebrew/lib/python3.9/site-packages (from highdicom) (1.4.0)
Requirement already satisfied: numpy>=1.19 in /opt/homebrew/lib/python3.9/site-packages (from highdicom) (1.22.1)
Requirement already satisfied: pillow-jpls>=1.0 in /opt/homebrew/lib/python3.9/site-packages (from highdicom) (1.1.0)
Requirement already satisfied: pydicom>=2.2.2 in /opt/homebrew/lib/python3.9/site-packages (from highdicom) (2.2.2)
Requirement already satisfied: pillow>=8.3 in /opt/homebrew/lib/python3.9/site-packages (from highdicom) (9.0.0)
Requirement already satisfied: pylibjpeg-openjpeg>=1.1 in /opt/homebrew/lib/python3.9/site-packages (from highdicom) (1.2.0)
Requirement already satisfied: pylibjpeg-libjpeg>=1.2 in /opt/homebrew/lib/python3.9/site-packages (from highdicom) (1.3.0)

>>> import highdicom
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/homebrew/lib/python3.9/site-packages/highdicom/__init__.py", line 4, in <module>
    from highdicom import legacy
  File "/opt/homebrew/lib/python3.9/site-packages/highdicom/legacy/__init__.py", line 4, in <module>
    from highdicom.legacy.sop import (
  File "/opt/homebrew/lib/python3.9/site-packages/highdicom/legacy/sop.py", line 18, in <module>
    from highdicom.frame import encode_frame
  File "/opt/homebrew/lib/python3.9/site-packages/highdicom/frame.py", line 6, in <module>
    import pillow_jpls  # noqa
  File "/opt/homebrew/lib/python3.9/site-packages/pillow_jpls/__init__.py", line 2, in <module>
    from .jpls_image_file import JplsImageFile, accept
  File "/opt/homebrew/lib/python3.9/site-packages/pillow_jpls/jpls_image_file.py", line 3, in <module>
    from . import _pycharls
ImportError: dlopen(/opt/homebrew/lib/python3.9/site-packages/pillow_jpls/_pycharls.cpython-39-darwin.so, 2): Symbol not found: __ZN3fmt2v87vformatENS0_17basic_string_viewIcEENS0_17basic_format_argsINS0_20basic_format_contextINS0_8appenderEcEEEE
  Referenced from: /opt/homebrew/lib/python3.9/site-packages/pillow_jpls/_pycharls.cpython-39-darwin.so
  Expected in: flat namespace
 in /opt/homebrew/lib/python3.9/site-packages/pillow_jpls/_pycharls.cpython-39-darwin.so

@elamathis18, I don't think that's related to this issue. Please follow this up at https://github.com/MGHComputationalPathology/highdicom -- there might already be an issue open there. If they say it's a Homebrew issue, then open a new issue here and we'll try to help you out.

In case someone still needs convincing:

This is the exact reason why DEPRECATION messages must not be presented to the wrong audience. inexperienced users think it's related to the error that follows the DEPRECATION message.

An example from another world, npm requests package deprecation message points to this issue:

request/request#3142
and now it has 2189 hidden items which are mostly error reports from users that got errors and followed the first link in the error report. one example bug report:
request/request#3142 (comment)

This also affects custom installs derived from Homebrew, by complicating the process even further.

Still mind-bending that Python maintainers actually propose "Patch Python if you want to a custom install scheme"

distutils.cfg file (and its various variants) has long been disfavoured by Python core maintainers, and the recommendation, if redistributors wish to provide custom install schemes, has been to directly patch distutils and sysconfig instead.

Install powerline git-status do not work any more. What can I do?

pip3 install powerline-gitstatus

DEPRECATION: Configuring installation scheme with distutils config files is deprecated and will no longer work in the near future. If you are using a Homebrew or Linuxbrew Python, please see discussion at https://github.com/Homebrew/homebrew-core/issues/76621

If there's nothing that can be done at this point, is there a way to silence the deprecation warnings, at least temporarily? I get many of these warnings when installing packages which is not ideal.

opens the link from the pip install
immediately sees 'nothing is expected to break until python 3.12'
closes tab

If there's nothing that can be done at this point, is there a way to silence the deprecation warnings, at least temporarily? I get many of these warnings when installing packages which is not ideal.

I suggest opening a pull request to make the changes you'd like to see at https://github.com/pypa/pip/blob/ec8edbf5df977bb88e1c777dd44e26664d81e216/src/pip/_internal/locations/__init__.py#L383-L392.

There is no way to silence this warning from Homebrew without patching pip, and we have a policy of not applying patches that aren't accepted upstream.

If there's nothing that can be done at this point, is there a way to silence the deprecation warnings, at least temporarily? I get many of these warnings when installing packages which is not ideal.

@iliyang The best solution for now is the one mentioned in this comment. However, you need to either pin python, or apply this patch every time Homebrew Python updates.

The best solution for now is to use Python 3.10, where we've already removed distutils.cfg. I worked with upstream to make the necessary changes in setuptools 60 to allow us to do that.

I'm evaluating what we'll do for Python 3.9 as it will not involve removing that file for compatibility reasons.

@Bo98 Oh, that's news to me... but very good news. So Homebrew Python 3.10 works nicely now, with no annoying warnings? Hopefully 3.10 can be become the default version soon, rather than keg-only.

The best solution for now is to use Python 3.10

@Bo98 Well that makes it easy to silence the warnings. Thanks for pointing that out.

For anyone wondering the progress of the migrating to python@3.10, it's being tracked in #87075

Error for running pip requests

byteoveflow@ANDRIs-MacBook-Pro project-staff % pip install requests
DEPRECATION: Configuring installation scheme with distutils config files is deprecated and will no longer work in the near future. If you are using a Homebrew or Linuxbrew Python, please see discussion at https://github.com/Homebrew/homebrew-core/issues/76621
Requirement already satisfied: requests in /opt/homebrew/lib/python3.9/site-packages (2.27.1)
Requirement already satisfied: certifi>=2017.4.17 in /opt/homebrew/lib/python3.9/site-packages (from requests) (2021.10.8)
Requirement already satisfied: charset-normalizer~=2.0.0 in /opt/homebrew/lib/python3.9/site-packages (from requests) (2.0.12)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /opt/homebrew/lib/python3.9/site-packages (from requests) (1.26.8)
Requirement already satisfied: idna<4,>=2.5 in /opt/homebrew/lib/python3.9/site-packages (from requests) (3.3)
DEPRECATION: Configuring installation scheme with distutils config files is deprecated and will no longer work in the near future. If you are using a Homebrew or Linuxbrew Python, please see discussion at https://github.com/Homebrew/homebrew-core/issues/76621

any clarfications here ?

Any way to disable this warning?

@Atcold See my above comment, which links to the solution (best for now).

The best solution for now is to use Python 3.10, where we've already removed distutils.cfg. I worked with upstream to make the necessary changes in setuptools 60 to allow us to do that.

I'm evaluating what we'll do for Python 3.9 as it will not involve removing that file for compatibility reasons.

if so, python -m pip -V shows 3.10, pip -V shows 3.9

Nothing is currently expected to break until Python 3.12

To hide this warning, for linux user:

brew pin python@3.9
find: /home/linuxbrew/.linuxbrew/lib/python3.9/site-packages/pip/_internal/locations/init.py

ref:

Looks like this was fixed in 3.10.2, by patching Python:

homebrew-core/Formula/python@3.10.rb

Line 62 in e3dbb8a

# Modify default sysconfig to match the brew install layout.

Yeah, I mentioned it earlier here.

It was middle ground we met, where the other half was working with the setuptools devs to implement some fixes on their side. You must be using setuptools 60 or later to have received those fixes. We're hoping to drop those patches altogether in the future however (no later than Python 3.12) when some changes to sysconfig hopefully land upstream. This is the condition on which we accepted those patches.

I've not however fixed Python 3.9 yet, which is why I've not closed this issue. Python 3.9 requires a bigger patch and the Python 3.10 patches have notable shortcomings that could break users' workflows, such as --prefix no longer working properly for setuptools. These are shortcomings that we cannot fix ourselves without work upstream (this is kinda the point raised earlier - this migration was forced far too early when the system has not matured enough yet). Fedora's Python shares the same issues, causing some downstream pain in several projects. We have relied on a hack to get at least pip working ok.

@pradyunsg

🚧 Never link to branches, share a permalink 🚧

see https://news.ycombinator.com/item?id=8046710
and https://help.github.com/articles/getting-permanent-links-to-files/
and https://docs.gitlab.com/ee/user/shortcuts.html#project-files
hit y
and perhaps for https://github.com/josephfrazier/octopermalinker (GitHub)

NOTE: edit your post and fix it with a permalink!

Can you help me how to fix it this problem
!pip install -q transformers
DEPRECATION: Configuring installation scheme with distutils config files is deprecated and will no longer work in the near future. If you are using a Homebrew or Linuxbrew Python, please see discussion at #76621

Sweet warning.

$ pip3.9 install justpy==0.1.5
On my macOS High Sierra I was able to installed justpy after I ran
pip3.9 install setuptools==56.0.0
but several DEPRECATION warnings were given

In pypa/distutils#152 (comment) and associated PR, I've drafted a change for distutils that may address the Python 3.9 concern within distutils itself (and would be released in a future release of Setuptools). Please take a look, help test and critique the approach. Thanks.

how to fixed this ?

how to fixed this ?

I had to literally go back to python3.9 and now I see this error, apparently the project I'm working on could not be upgraded, how do we fix this?

I am getting lots of errors on homebrew because of this. Is there a way to hide these warnings on mac meanwhile?

The solution to this problem I understand is to upgrade 3.9->3.10+ on it?

Related: #152017

@Bo98 what do you think about closing this issue? I feel that we will never « fix » this in Python@3.9 (and to be fair, that’s totally OK).

Most of our formulae have migrated to newer Python versions and Python@3.9 downloads are slowly decreasing. The EOL date is 2025-10, so users will have to survive with current state until then.

Yeh, let's close, thanks @carlocab @iMichka and everyone else who helped here!