aws/aws-cli

"No package metadata" during wheel build from source

Closed this issue · 12 comments

dgilman commented

Describe the bug

The issue here is that the wheel build of awscli v2 from a source distribution is now failing with this exception after updating to the latest prompt_toolkit 3.0.51. Note that this version of prompt_toolkit now uses pyproject.toml for its build system. The output from awscli2's build is:

$ python3.13 -m build --no-isolation --wheel --outdir my_awscli_dir

* Getting build dependencies for wheel...
* Building wheel...
Traceback (most recent call last):
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.13/lib/python3.13/importlib/metadata/__init__.py", line 407, in from_name
    return next(iter(cls.discover(name=name)))
StopIteration

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.13/lib/python3.13/site-packages/pyproject_hooks/_in_process/_in_process.py", line 389, in <module>
    main()
    ~~~~^^
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.13/lib/python3.13/site-packages/pyproject_hooks/_in_process/_in_process.py", line 373, in main
    json_out["return_val"] = hook(**hook_input["kwargs"])
                             ~~~~^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.13/lib/python3.13/site-packages/pyproject_hooks/_in_process/_in_process.py", line 280, in build_wheel
    return _build_backend().build_wheel(
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        wheel_directory, config_settings, metadata_directory
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/opt/local/var/macports/build/_Users_david_macports-ports-local_python_py-awscli2/py313-awscli2/work/aws-cli-2.26.5/backends/pep517.py", line 65, in build_wheel
    _inject_wheel_extras(os.path.join(wheel_directory, whl_filename))
    ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/local/var/macports/build/_Users_david_macports-ports-local_python_py-awscli2/py313-awscli2/work/aws-cli-2.26.5/backends/pep517.py", line 191, in _inject_wheel_extras
    _build_and_inject_ac_index(BUILD_DIR, extracted_wheel_dir)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/local/var/macports/build/_Users_david_macports-ports-local_python_py-awscli2/py313-awscli2/work/aws-cli-2.26.5/backends/pep517.py", line 219, in _build_and_inject_ac_index
    ac_index_build_name = _build_ac_index(build_dir)
  File "/opt/local/var/macports/build/_Users_david_macports-ports-local_python_py-awscli2/py313-awscli2/work/aws-cli-2.26.5/backends/pep517.py", line 230, in _build_ac_index
    from awscli.autocomplete.generator import generate_index
  File "/opt/local/var/macports/build/_Users_david_macports-ports-local_python_py-awscli2/py313-awscli2/work/aws-cli-2.26.5/awscli/autocomplete/generator.py", line 17, in <module>
    from awscli import clidriver
  File "/opt/local/var/macports/build/_Users_david_macports-ports-local_python_py-awscli2/py313-awscli2/work/aws-cli-2.26.5/awscli/clidriver.py", line 52, in <module>
    from awscli.autoprompt.core import AutoPromptDriver
  File "/opt/local/var/macports/build/_Users_david_macports-ports-local_python_py-awscli2/py313-awscli2/work/aws-cli-2.26.5/awscli/autoprompt/core.py", line 17, in <module>
    from awscli.autoprompt.prompttoolkit import PromptToolkitPrompter
  File "/opt/local/var/macports/build/_Users_david_macports-ports-local_python_py-awscli2/py313-awscli2/work/aws-cli-2.26.5/awscli/autoprompt/prompttoolkit.py", line 18, in <module>
    from prompt_toolkit.application import Application
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.13/lib/python3.13/site-packages/prompt_toolkit/__init__.py", line 32, in <module>
    __version__ = metadata.version("prompt_toolkit")
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.13/lib/python3.13/importlib/metadata/__init__.py", line 987, in version
    return distribution(distribution_name).version
           ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.13/lib/python3.13/importlib/metadata/__init__.py", line 960, in distribution
    return Distribution.from_name(distribution_name)
           ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.13/lib/python3.13/importlib/metadata/__init__.py", line 409, in from_name
    raise PackageNotFoundError(name)
importlib.metadata.PackageNotFoundError: No package metadata was found for prompt_toolkit

This appears to be the same issue brought up on the arch linux forums that also sometimes crops up with kubectl.

I am not sure exactly where the issue lies but I bring it up here because after digging into the issue a bit, I found that the issue happens when awscli2's backends/pep517.py is the backend_path used by pyproject_hooks to find the package metadata. In other words, if I:

  • take the bug fix in pyproject_metadata in this PR
  • but change how it instantiates the DistributionFinder.Context to DistributionFinder.Context(name=name) - or in other words, keep it from using awscli2's backends/pep517 as the backend_path, it correctly finds the import and the awscli2 build works. But that is clearly a bogus fix, it just disables your backends/pep517.py and I don't think upstream is supposed to do that.

This correlates with what people are saying on the Arch forums, and what I've seen locally: if I run a python REPL and just do metadata.version("prompt_toolkit") in there it finds the package OK, so there is something about how awscli2's backends/pep517.py is injected into the importlib mechanisms that makes the build tooling unable to find system packages.

I don't know enough about the pep517 build stuff to know conclusively where the problem is, maybe it is with prompt_toolkit, maybe it is with pyproject_hooks, maybe it is with your backends/ dir. But it is the combination of all three that is glitching this out.

Regression Issue

  • Select this option if this issue appears to be a regression.

Expected Behavior

python -m build --no-isolation --wheel --outdir blah should build the wheel

Current Behavior

See exception above

Reproduction Steps

Run python -m build --no-isolation --wheel --outdir blah on python 3.13 with prompt_toolkit 3.0.51 installed.

Possible Solution

No response

Additional Information/Context

No response

CLI version used

n/a

Environment details (OS name and version, etc.)

macOS 15.4, python 3.13

steven-esser commented

Yep, experiencing this as well.

  • 👍2
evanstucker-hates-2fa commented

Temporary workaround for people using Arch who still have the previous package cached - downgrade python-prompt_toolkit:

sudo pacman -U /var/cache/pacman/pkg/python-prompt_toolkit-3.0.50-1-any.pkg.tar.zst
  • 👍15
adev-code commented

Hi @dgilman and all, thanks for bringing this up. We only support toolkit version from 3.0.24 to 3.0.38

aws-cli/pyproject.toml

Line 42 in f437589

"prompt-toolkit>=3.0.24,<3.0.39",
. It seems that the version you are using is out of range. Please downgrade to a version that we support. Let me know if there are an questions.

Please see @evanstucker-hates-2fa open issue prompt-toolkit/python-prompt-toolkit#1988 to the right repo.

  • 👎17
dgilman commented

I understand it is not supported. However, at some point, you’ll be bumping the package version yourself and run into the same issue. And although I don’t know exactly where the pep517 build tooling is breaking down evidence suggests that prompt_toolkit is a red herring because its metadata can be successfully retrieved by importlib when not running within the awscli2 build process.

  • 👍2
tema-mazy commented

macport aws is broken .

$aws
Traceback (most recent call last):
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.13/lib/python3.13/importlib/metadata/init.py", line 407, in from_name
return next(iter(cls.discover(name=name)))
StopIteration

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/opt/local/bin/aws", line 19, in
import awscli.clidriver
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.13/lib/python3.13/site-packages/awscli/clidriver.py", line 73, in
from awscli.autoprompt.core import AutoPromptDriver
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.13/lib/python3.13/site-packages/awscli/autoprompt/core.py", line 16, in
from awscli.autoprompt.prompttoolkit import PromptToolkitPrompter
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.13/lib/python3.13/site-packages/awscli/autoprompt/prompttoolkit.py", line 18, in
from prompt_toolkit.application import Application
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.13/lib/python3.13/site-packages/prompt_toolkit/init.py", line 32, in
version = metadata.version("prompt_toolkit")
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.13/lib/python3.13/importlib/metadata/init.py", line 987, in version
return distribution(distribution_name).version
~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.13/lib/python3.13/importlib/metadata/init.py", line 960, in distribution
return Distribution.from_name(distribution_name)
~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.13/lib/python3.13/importlib/metadata/init.py", line 409, in from_name
raise PackageNotFoundError(name)
importlib.metadata.PackageNotFoundError: No package metadata was found for prompt_toolkit

  • 👍4
chadgeary commented

alpine's aws-cli package is also broken

aws version
Traceback (most recent call last):
  File "/usr/lib/python3.12/importlib/metadata/__init__.py", line 397, in from_name
    return next(cls.discover(name=name))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
StopIteration

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/bin/aws", line 19, in <module>
    import awscli.clidriver
  File "/usr/lib/python3.12/site-packages/awscli/clidriver.py", line 67, in <module>
    from awscli.autoprompt.core import AutoPromptDriver
  File "/usr/lib/python3.12/site-packages/awscli/autoprompt/core.py", line 16, in <module>
    from awscli.autoprompt.prompttoolkit import PromptToolkitPrompter
  File "/usr/lib/python3.12/site-packages/awscli/autoprompt/prompttoolkit.py", line 18, in <module>
    from prompt_toolkit.application import Application
  File "/usr/lib/python3.12/site-packages/prompt_toolkit/__init__.py", line 32, in <module>
    __version__ = metadata.version("prompt_toolkit")
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/importlib/metadata/__init__.py", line 889, in version
    return distribution(distribution_name).version
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/importlib/metadata/__init__.py", line 862, in distribution
    return Distribution.from_name(distribution_name)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/importlib/metadata/__init__.py", line 399, in from_name
    raise PackageNotFoundError(name)
importlib.metadata.PackageNotFoundError: No package metadata was found for prompt_toolkit
amberkushwaha commented

this issue here is that the main source file interpretation has been manipulated in the coding file distribution.

project0 commented

I am pretty certain the problem is with the aws-cli packaging. Executing the affected python code manually with global installed prompt toolkit dependency (using arch) the import works as expected.

thomas-roos commented

+1 issue on meta-aws (yocto)

hssyoo commented

Hi everyone, this issue was resolved in #9552 and released in version 2.27.56. Upgrading to the latest version should help. Please let us know if you're experiencing other issues.

github-actions commented

This issue is now closed. Comments on closed issues are hard for our team to see.
If you need more assistance, please open a new issue that references this one.