python-poetry/poetry

1.1.3 regression. poetry exports wrong constraints

Closed this issue · 5 comments

apollo13 commented
  • I am on the latest Poetry version.
  • I have searched the issues of this repo and believe that this is not a duplicate.
  • If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

Issue

The linked pyproject.toml generates a broken requirements.txt (it wrongly applies sys_platform=darwin). But first off lets see the graph:

pyinstaller 4.0 PyInstaller bundles a Python application and all its dependencies into a single package.
├── altgraph *
├── macholib >=1.8
│   └── altgraph >=0.15 
├── pefile >=2017.8.1
│   └── future * 
├── pyinstaller-hooks-contrib >=2020.6
└── pywin32-ctypes >=0.2.0

as we can see the toplevel dependency pyinstaller directly requires althgraph and then macholib also requires altgraph.

Looking at poetry.lock we see:

[[package]]
name = "pyinstaller"
version = "4.0"
description = "PyInstaller bundles a Python application and all its dependencies into a single package."
category = "main"
optional = false
python-versions = "*"

[package.dependencies]
altgraph = "*"
macholib = {version = ">=1.8", markers = "sys_platform == \"darwin\""}
pefile = {version = ">=2017.8.1", markers = "sys_platform == \"win32\""}
pyinstaller-hooks-contrib = ">=2020.6"
pywin32-ctypes = {version = ">=0.2.0", markers = "sys_platform == \"win32\""}

[[package]]
name = "macholib"
version = "1.14"
description = "Mach-O header analysis and editing"
category = "main"
optional = false
python-versions = "*"

[package.dependencies]
altgraph = ">=0.15"

This tells us that pyinstaller depends on macholib on darwin systems. The export looks like this:

altgraph==0.17; sys_platform == "darwin" \
    --hash=sha256:c623e5f3408ca61d4016f23a681b9adb100802ca3e3da5e718915a9e4052cebe \
    --hash=sha256:1f05a47122542f97028caf78775a095fbe6a2699b5089de8477eb583167d69aa
future==0.18.2; python_version >= "2.6" and python_full_version < "3.0.0" and sys_platform == "win32" or python_full_version >= "3.3.0" and sys_platform == "win32" \
    --hash=sha256:b1bead90b70cf6ec3f0710ae53a525360fa360d306a86583adc6bf83a4db537d
macholib==1.14; sys_platform == "darwin" \
    --hash=sha256:c500f02867515e6c60a27875b408920d18332ddf96b4035ef03beddd782d4281 \
    --hash=sha256:0c436bc847e7b1d9bda0560351bf76d7caf930fb585a828d13608839ef42c432
pefile==2019.4.18; sys_platform == "win32" \
    --hash=sha256:a5d6e8305c6b210849b47a6174ddf9c452b2888340b8177874b862ba6c207645
pyinstaller-hooks-contrib==2020.9 \
    --hash=sha256:a5fd45a920012802e3f2089e1d3501ef2f49265dfea8fc46c3310f18e3326c91 \
    --hash=sha256:c382f3ac1a42b45cfecd581475c36db77da90e479b2f5bcb6d840d21fa545114
pyinstaller==4.0 \
    --hash=sha256:970beb07115761d5e4ec317c1351b712fd90ae7f23994db914c633281f99bab0
pywin32-ctypes==0.2.0; sys_platform == "win32" \
    --hash=sha256:24ffc3b341d457d48e8922352130cf2644024a4ff09762a2261fd34c36ee5942 \
    --hash=sha256:9dc2d991b3479cc2df15930958b674a48a227d5361d413827a4cfd0b5876fc98

Here altgraph is incorrectly labeled as darwin, propbably due to the fact that macholib is only required on darwin and as such the constraints trickles down. This makes sense but should not happen if there is another dependency on the same package.

jonapich commented

I have the same problem with the six package on Windows:

[tool.poetry]
name = "test"
version = "0.4.0"
description = ""
authors = [""]

[tool.poetry.dependencies]
python = ">=3.6"
junit-xml = "*"
poetry = "^1.1.4"

[build-system]
requires = ["poetry_core>=1.0.0"]
build-backend = "poetry.core.masonry.api"

In this scenario, junit-xml requires six on all platforms.

But poetry <- keyring <- secretstorage <- cryptography <- six markers are merged to the six DependencyPackage instance here which transforms six into a linux-only package, because keyring needs secretstorage on linux only.

sinoroc commented

I feel like I start recognizing a pattern here... There is at least 1 (maybe 2) other issue like that, where something similar seems to happen. Two unrelated dependencies on the same library are incorrectly merged.

bneijt commented

I don't understand why markers should be propagated up the dependency tree. It looks like

poetry/poetry/packages/locker.py

Line 514 in de0b32c

if not dependency.marker.is_any():
suggests that a marker from a dependency is a constraint for a package. But if the dependency is optional then should it even matter?

apollo13 commented

@jonapich My case got fixed by the following diff:

diff --git a/poetry/packages/locker.py b/poetry/packages/locker.py
index 48d68e95..f06d36d6 100644
--- a/poetry/packages/locker.py
+++ b/poetry/packages/locker.py
@@ -269,9 +269,9 @@ class Locker(object):
             if key not in nested_dependencies:
                 nested_dependencies[key] = requirement
             else:
-                nested_dependencies[key].marker = nested_dependencies[
-                    key
-                ].marker.intersect(requirement.marker)
+                nested_dependencies[key].marker = nested_dependencies[key].marker.union(
+                    requirement.marker
+                )
 
         return cls.__walk_dependency_level(
             dependencies=next_level_dependencies,

Could you try if this also fixes your case?

github-actions commented

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.