aspect-build/rules_py

[Bug]: `--build_python_zip` seems to not work in py_binary

Opened this issue · 3 comments

What happened?

When building a py_binary with --build_python_zip specified, expected .zip artifact is not generated.

Version

Development (host) and target OS/architectures:
Ubuntu 20.04

Output of bazel --version:
bazel 7.0.0 (bazelisk)

Version of the Aspect rules, or other relevant rules from your
WORKSPACE or MODULE.bazel file:
aspect_rules_py, version: 0.5.0
rules_python, version: 0.22.0

Language(s) and/or frameworks involved:
Python

How to reproduce

# MODULE.bazel

bazel_dep(name = "aspect_rules_py", version = "0.5.0")

bazel_dep(name = "rules_python", version = "0.22.0", dev_dependency = True)

python = use_extension("@rules_python//python/extensions:python.bzl", "python")
python.toolchain(
    name = "python3",
    configure_coverage_tool = True,
    python_version = "3.11",
)
use_repo(python, "python3_toolchains")

register_toolchains(
    "@python3_toolchains//:all",
)
# BUILD

load("@aspect_rules_py//py:defs.bzl", "py_binary")

py_binary(
    name = "script",
    srcs = ["script.py"],
    main = "script.py",
)

Running with: bazel build //:script --build_python_zip



### Any other information?

_No response_

We had a design discussion about this today, but we don't have enough requirements.

I'm not sure that the Bazel implementation for --build_python_zip produces a file that follows any existing specification. It was written long ago in https://bazel-review.googlesource.com/c/bazel/+/4244 and there's no thread there about how the structure of the file was determined. I'm very hesitant to build an unspecified thing that just emulates this ancient Bazel feature, as it exposes us to a ton of bugs if we make wrong choices. We wouldn't be able to reply to such bugs with "we produce the following well-understood output, so you can reproduce that problem outside of Bazel"

We could start from first-principles and follow https://peps.python.org/pep-0441/ - in this case rules_py could produce a zipapp following that specification that includes only the python files in your application (and the runfiles tree for their data dependencies)
However that spec doesn't include an interpreter, or third-party packages. It's typically intended for deployment to some environment where you already have those things (e.g. a docker container where the base included python and pip install is run as a step, or maybe a remote orchestrator service or lambda environment whose docs say "upload your application code")

We could offer something different - a PEX file is a well-understood "self-extracting executable" for Python. rules_py could produce something identical to https://www.pantsbuild.org/2.20/docs/python/overview/pex and this would be usable in more cases.

If there is a self executing python file, that would seem like a upgrade and much better then zip.. Others can tell me if I am wrong, but currently, the zip is just the best we got right now, sorta deal. I would much prefer to just execute a file like most other code can do.

@alexeagle Thanks for the reply. We are using a docker environment so both .pyz and .pex satisfy our use case. I'm not faimliar with Bazel implementation details so please follow your decision.