`py` preprocessor selectors not getting updated with variant versions

Question

`py` preprocessor selectors not getting updated with variant versions

shoemakerdr opened this issue 2 years ago · 2 comments

Hello!

When I try to build a package with multiple python variants, the py selector is using my environment python (3.8.12) rather than the one defined in the current variant:

Env info

(base) $ uname -o
GNU/Linux

(base) $ mamba --version
mamba 1.0.0
conda 22.9.0

(base) $ python --version
Python 3.8.12

(base) $ conda build --version
conda-build 3.22.0

(base) $ boa --version
0.13.0

Reproducible example

# conda-recipe/recipe.yaml
package:
  name: example
  version: "0.1.0"

source:
  path: ..

build:
  script: "$PYTHON setup.py install --single-version-externally-managed --record=record.txt"
  number: 0

requirements:
  host:
    - python {{ python }}
    - setuptools *
  run:
    - python {{ python }}
    - sel(py<38): importlib_metadata
    - sel(py==38): pandas

# conda-recipe/conda_build_config.yaml
python:
    - 3.7
    - 3.8
    - 3.9

Expected behavior

The python 3.7 variant should include importlib_metadata but not pandas, the 3.8 variant should include pandas but not importlib_metadata, and the 3.9 variant should include neither.

`boa render` output

(base) $ boa render conda-recipe
...
           Output: example
┏━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┓
┃ Package         ┃ Variant versions ┃
┡━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━┩
│ target_platform │ linux-64         │
│ python          │ 3.7              │
│                 │ 3.8              │
└─────────────────┴──────────────────┘
                   Output: example 0.1.0 BN: 0
                          Variant: 3.7
                             Build:

             ╷                     ╷          ╷       ╷
  Dependency │ Version requirement │ Selected │ Build │ Channel
 ════════════╪═════════════════════╪══════════╪═══════╪═════════
             │                     │          │       │
  Host       │                     │          │       │
  python     │ 3.7                 │          │       │
  setuptools │ *                   │          │       │
             │                     │          │       │
  Run        │                     │          │       │
  python     │                     │          │       │
  pandas     │                     │          │       │
             ╵                     ╵          ╵       ╵
                   Output: example 0.1.0 BN: 0
                          Variant: 3.8
                             Build:

             ╷                     ╷          ╷       ╷
  Dependency │ Version requirement │ Selected │ Build │ Channel
 ════════════╪═════════════════════╪══════════╪═══════╪═════════
             │                     │          │       │
  Host       │                     │          │       │
  python     │ 3.8                 │          │       │
  setuptools │ *                   │          │       │
             │                     │          │       │
  Run        │                     │          │       │
  python     │                     │          │       │
  pandas     │                     │          │       │
             ╵                     ╵          ╵       ╵
                   Output: example 0.1.0 BN: 0
                          Variant: 3.9
                             Build:

             ╷                     ╷          ╷       ╷
  Dependency │ Version requirement │ Selected │ Build │ Channel
 ════════════╪═════════════════════╪══════════╪═══════╪═════════
             │                     │          │       │
  Host       │                     │          │       │
  python     │ 3.9                 │          │       │
  setuptools │ *                   │          │       │
             │                     │          │       │
  Run        │                     │          │       │
  python     │                     │          │       │
  pandas     │                     │          │       │
             ╵                     ╵          ╵       ╵

Answer 1 · 2023-01-30T11:04:11.000Z

Im sorry, this comment was previously misplaced in #308

Hi,

I'm encountering ( perhaps ) a similar issue.

When boa building for multiple py versions, the variants versions are not "inserted" to the built packages requirements.

env info -

(base) > $ mamba list "(boa|mamba|conda)"
# packages in environment at /Users/harelos/mambaforge:
#
# Name                    Version                   Build  Channel
boa                       0.14.0             pyhd8ed1ab_0    conda-forge
conda                     22.11.1         py310h2ec42d9_1    conda-forge
conda-build               3.23.3          py310h2ec42d9_0    conda-forge
conda-package-handling    1.8.1           py310h1961e1f_1    conda-forge
conda-souschef            2.2.3              pyhd8ed1ab_0    conda-forge
libmamba                  1.1.0                h8b63968_2    conda-forge
libmambapy                1.1.0           py310hb15139c_2    conda-forge
mamba                     1.1.0           py310h6bde348_2    conda-forge

recipe.yaml -

context:
  name: hera-workflows
  version: 4.3.0

package:
  name: '{{ name|lower }}'
  version: '{{ version }}'

source:
  url: https://pypi.io/packages/source/{{ name[0] }}/{{ name }}/hera_workflows-{{ version }}.tar.gz
  sha256: 30976fe13de04d803d430825721532921f5b18edbd90b861e5d69b0ba8123859

build:
  script: '{{ PYTHON }} -m pip install . -vv'
  number: 0

requirements:
  build:
    - python {{ python }}
  host:
    - python
    - pip
    - poetry
  run:
    - python
    - argo-workflows ==6.3.5
    - certifi >=2021.10.8
    - python-dateutil >=2.8.2
    - pytz >=2021.3
    - urllib3 >=1.26.8
    - typing-extensions >=3.10.0

test:
  imports:
    - hera
  commands:
    - pip check
  requires:
    - pip

about:
  home: https://github.com/argoproj-labs/hera-workflows
  summary: Hera is a Python framework for constructing and submitting Argo Workflows. The main goal of Hera is to make Argo Workflows more accessible by abstracting away some setup that is typically necessary for constructing Argo workflows.
  license: MIT
  license_file: LICENSE

extra:
  recipe-maintainers:
    - harelos

conda_build_config.yaml -

python:
  - 3.9
  - 3.10

Build + Build results -

(base) > $ boa build ./ --output-folder /tmp/boa-build --no-test 
[...]

(base) > $ mamba search "hera-workflows" -c /tmp/boa-build --override-channels --info

Loading channels: done
hera-workflows 4.3.0 py310h8ff6bd7_0
------------------------------------
file name   : hera-workflows-4.3.0-py310h8ff6bd7_0.tar.bz2
name        : hera-workflows
version     : 4.3.0
build       : py310h8ff6bd7_0
build number: 0
size        : 104 KB
subdir      : osx-64
url         : file:///tmp/boa-build/osx-64/hera-workflows-4.3.0-py310h8ff6bd7_0.tar.bz2
md5         : 5ce2329a058821da13de8e60521b897b
timestamp   : 2022-12-27 08:07:42 UTC
dependencies: 
  - argo-workflows ==6.3.5
  - certifi >=2021.10.8
  - python
  - python-dateutil >=2.8.2
  - python_abi 3.10.* *_cp310
  - pytz >=2021.3
  - typing-extensions >=3.10.0
  - urllib3 >=1.26.8


hera-workflows 4.3.0 py39h59b1b0c_0
-----------------------------------
file name   : hera-workflows-4.3.0-py39h59b1b0c_0.tar.bz2
name        : hera-workflows
version     : 4.3.0
build       : py39h59b1b0c_0
build number: 0
size        : 102 KB
subdir      : osx-64
url         : file:///tmp/boa-build/osx-64/hera-workflows-4.3.0-py39h59b1b0c_0.tar.bz2
md5         : 1988dc937d2913412e8e9dbd2f05fc2b
timestamp   : 2022-12-27 08:07:16 UTC
dependencies: 
  - argo-workflows ==6.3.5
  - certifi >=2021.10.8
  - python
  - python-dateutil >=2.8.2
  - python_abi 3.9.* *_cp39
  - pytz >=2021.3
  - typing-extensions >=3.10.0
  - urllib3 >=1.26.8

Whereas, when running conda-build for the following meta.yaml -

{% set name = "hera-workflows" %}
{% set version = "4.3.0" %}

package:
  name: {{ name|lower }}
  version: {{ version }}

source:
  url: https://pypi.io/packages/source/{{ name[0] }}/{{ name }}/hera_workflows-{{ version }}.tar.gz
  sha256: 30976fe13de04d803d430825721532921f5b18edbd90b861e5d69b0ba8123859

build:
  script: {{ PYTHON }} -m pip install . -vv
  number: 0

requirements:
  build:
    - python {{ python }}
  host:
    - python
    - pip
    - poetry
  run:
    - python
    - argo-workflows ==6.3.5
    - certifi >=2021.10.8
    - python-dateutil >=2.8.2
    - pytz >=2021.3
    - urllib3 >=1.26.8
    - typing-extensions >=3.10.0

test:
  imports:
    - hera
  commands:
    - pip check
  requires:
    - pip

about:
  home: https://github.com/argoproj-labs/hera-workflows
  summary: Hera is a Python framework for constructing and submitting Argo Workflows. The main goal of Hera is to make Argo Workflows more accessible by abstracting away some setup that is typically necessary for constructing Argo workflows.
  license: MIT
  license_file: LICENSE

extra:
  recipe-maintainers:
    - harelos

Build + Build results -

(base) > $ conda-mambabuild ./meta.yaml --output-folder /tmp/conda-build-out/ 

[...]

(base) > $ mamba search -c /tmp/conda-build-out --override-channels --info

Loading channels: done
hera-workflows 4.3.0 py310_0
----------------------------
file name   : hera-workflows-4.3.0-py310_0.tar.bz2
name        : hera-workflows
version     : 4.3.0
build       : py310_0
build number: 0
size        : 104 KB
license     : MIT
subdir      : osx-64
url         : file:///tmp/conda-build-out/osx-64/hera-workflows-4.3.0-py310_0.tar.bz2
md5         : 4690f05089a053ce892448ee27113e06
timestamp   : 2022-12-27 08:03:39 UTC
dependencies: 
  - argo-workflows 6.3.5
  - certifi >=2021.10.8
  - python >=3.10,<3.11.0a0
  - python-dateutil >=2.8.2
  - python_abi 3.10.* *_cp310
  - pytz >=2021.3
  - typing-extensions >=3.10.0
  - urllib3 >=1.26.8


hera-workflows 4.3.0 py39_0
---------------------------
file name   : hera-workflows-4.3.0-py39_0.tar.bz2
name        : hera-workflows
version     : 4.3.0
build       : py39_0
build number: 0
size        : 103 KB
license     : MIT
subdir      : osx-64
url         : file:///tmp/conda-build-out/osx-64/hera-workflows-4.3.0-py39_0.tar.bz2
md5         : 53346e80d80b6d7d2ecb679d5fb6ef22
timestamp   : 2022-12-27 08:04:28 UTC
dependencies: 
  - argo-workflows 6.3.5
  - certifi >=2021.10.8
  - python >=3.9,<3.10.0a0
  - python-dateutil >=2.8.2
  - python_abi 3.9.* *_cp39
  - pytz >=2021.3
  - typing-extensions >=3.10.0
  - urllib3 >=1.26.8

Thanks,

Harel

Answer 2 · 2023-12-18T21:10:11.000Z

Bumping this because I'm still seeing this issue with the latest boa release (as of right now, 0.16.0)-- though now the selectors don't seem to be working at all.

Env info

Reproducible example

Expected behavior

boa render output

`boa render` output