The scie-pants caching for a given Pants version is only per build root and not per user.

Question

The scie-pants caching for a given Pants version is only per build root and not per user.

jsirois opened this issue 2 years ago · 0 comments

The issue is the configure scie binding which takes the build root as an input arg which gets mixed into its binding command fingerprint.

Looks like:

$ rm -rf ~/.cache/nce
$ mkdir one two
$ echo -e '[GLOBAL]\npants_version = "2.15.0"' > one/pants.toml
$ cp one/pants.toml two/
$ cd one

$ pants -V
Bootstrapping Pants 2.15.0 using cpython 3.9.15
Installing pantsbuild.pants==2.15.0 into a virtual environment at /home/jsirois/.cache/nce/d036c571b74670fb7cccdb2ee973b51625c6419b29dd5e2e5390d6267b873904/bindings/venvs/2.15.0
New virtual environment successfully created at /home/jsirois/.cache/nce/d036c571b74670fb7cccdb2ee973b51625c6419b29dd5e2e5390d6267b873904/bindings/venvs/2.15.0.
07:59:31.56 [INFO] Initializing scheduler...
07:59:31.67 [INFO] Scheduler initialized.
07:59:31.69 [WARN] Please either set `enabled = true` in the [anonymous-telemetry] section of pants.toml to enable sending anonymous stats to the Pants project to aid development, or set `enabled = false` to disable it. No telemetry sent for this run. An explicit setting will get rid of this message. See https://www.pantsbuild.org/v2.15/docs/anonymous-telemetry for details.
2.15.0
$ cd ../two/

$ RUST_LOG=trace pants -V
[DEBUG TimerFinished] lift::assemble(), Elapsed=4.362µs
[DEBUG TimerFinished] fingerprint::digest(), Elapsed=16.92µs
[DEBUG TimerFinished] lift::load_scie(), Elapsed=131.89µs
[TRACE jump] Loaded lift manifest from /home/jsirois/bin/pants:
...
[DEBUG TimerFinished] jump::load_dotenv, Elapsed=21.099µs
[DEBUG TimerFinished] Context::new(), Elapsed=13.974µs
[DEBUG jump::atomic] The atomic file at /home/jsirois/.cache/nce/0a49389f3d6f1abaddbd4b70b1803881e530e3dfcd0207bb4e6c2e12550a8009/scie-pants.bin has already been established.
[DEBUG TimerFinished] installer::unpack_blob(), Elapsed=37.43µs
[DEBUG TimerFinished] Installer::install(), Elapsed=50.961µs
[TRACE jump] Prepared Process {
...
[DEBUG TimerFinished] jump::prepare_boot(), Elapsed=1.016894ms
[DEBUG TimerFinished] BuildRoot::find(), Elapsed=8.936µs
[DEBUG TimerFinished] PantsConfig::parse(), Elapsed=34.589µs
[INFO  scie_pants] Found Pants build root at Some("/home/jsirois/support/scie-pants/Tobias/two")
[INFO  scie_pants] The required Pants version is Some("2.15.0")
[DEBUG TimerFinished] scie-pants::get_pants_process(), Elapsed=112.436µs
[TRACE scie_pants] Launching: Process {
...
[DEBUG TimerFinished] lift::assemble(), Elapsed=3.128µs
[DEBUG TimerFinished] fingerprint::digest(), Elapsed=15.608µs
[DEBUG TimerFinished] lift::load_scie(), Elapsed=114.7µs
[TRACE jump] Loaded lift manifest from /home/jsirois/bin/pants:
...
[DEBUG TimerFinished] jump::load_dotenv, Elapsed=8.323µs
[DEBUG TimerFinished] Context::new(), Elapsed=32.71µs
[DEBUG TimerFinished] Process::fingerprint(), Elapsed=25.039µs
[TRACE jump::context] Installing boot binding Binding {
        target: "/home/jsirois/.cache/nce/d036c571b74670fb7cccdb2ee973b51625c6419b29dd5e2e5390d6267b873904/locks/configure-46b2844906bc133ebcffbba98bca28e79d027ae7144e7944094b01461c00734b",
        process: Process {
            env: EnvVars {
                vars: [
                    Default(
                        (
                            "PANTS_VERSION_PROMPT_SALT",
                            "",
                        ),
                    ),
                    RemoveMatching(
                        ComparableRegex(
                            PEX_.*,
                        ),
                    ),
                    Replace(
                        (
                            "PEX_PYTHON_PATH",
                            "/home/jsirois/.cache/nce/cdc3a4cfddcd63b6cebdd75b14970e02d8ef0ac5be4d350e57ab5df56c19e85e/cpython-3.9.15+20221106-x86_64-unknown-linux-gnu-install_only.tar.gz/python/bin/python3.9",
                        ),
                    ),
                    Replace(
                        (
                            "PEX_ROOT",
                            "/home/jsirois/.cache/nce/d036c571b74670fb7cccdb2ee973b51625c6419b29dd5e2e5390d6267b873904/bindings/pex_root",
                        ),
                    ),
                ],
            },
            exe: "/home/jsirois/.cache/nce/cdc3a4cfddcd63b6cebdd75b14970e02d8ef0ac5be4d350e57ab5df56c19e85e/cpython-3.9.15+20221106-x86_64-unknown-linux-gnu-install_only.tar.gz/python/bin/python3.9",
            args: [
                "/home/jsirois/.cache/nce/04c117900eb9bf6005a70e25aa2dfe83ac94d648920c65b356c79cf8efdf48dc/tools.pex",
                "configure-pants",
                "--ptex-path",
                "/home/jsirois/.cache/nce/fd8ceac7b3026655b34b2c80997c18caeb7e8d64a9dd08c0340a5ced92f947d3/ptex",
                "--pants-version",
                "2.15.0",
                "--pants-sha",
                "",
                "--pants-config",
                "/home/jsirois/support/scie-pants/Tobias/two/pants.toml",
                "--github-api-bearer-token",
                "",
                "/home/jsirois/.cache/nce/d036c571b74670fb7cccdb2ee973b51625c6419b29dd5e2e5390d6267b873904/bindings",
            ],
        },
    }
[DEBUG jump::atomic] The atomic file at /home/jsirois/.cache/nce/fd8ceac7b3026655b34b2c80997c18caeb7e8d64a9dd08c0340a5ced92f947d3/ptex has already been established.
[DEBUG TimerFinished] installer::unpack_blob(), Elapsed=14.117µs
[DEBUG jump::atomic] The atomic file at /home/jsirois/.cache/nce/04c117900eb9bf6005a70e25aa2dfe83ac94d648920c65b356c79cf8efdf48dc/tools.pex has already been established.
[DEBUG TimerFinished] installer::unpack_blob(), Elapsed=30.394µs
[DEBUG jump::atomic] The atomic directory at /home/jsirois/.cache/nce/cdc3a4cfddcd63b6cebdd75b14970e02d8ef0ac5be4d350e57ab5df56c19e85e/cpython-3.9.15+20221106-x86_64-unknown-linux-gnu-install_only.tar.gz has already been established.
[DEBUG TimerFinished] installer::unpack_archive(), Elapsed=13.66µs
[DEBUG TimerFinished] Installer::install(), Elapsed=74.615µs
[DEBUG TimerFinished] Process::fingerprint(), Elapsed=25.008µs
[DEBUG jump::atomic] The atomic file at /home/jsirois/.cache/nce/d036c571b74670fb7cccdb2ee973b51625c6419b29dd5e2e5390d6267b873904/locks/install-d898c088a11d544cc4d38850bd9df88b964ef4aaf57b9bb74d64c5fc9f6076d5 has already been established.
[DEBUG TimerFinished] Installer::install(), Elapsed=320ns
[TRACE jump] Prepared Process {
...
[DEBUG TimerFinished] jump::prepare_boot(), Elapsed=1.099229286s
08:02:19.42 [INFO] Initializing scheduler...
08:02:19.51 [INFO] Scheduler initialized.
08:02:19.54 [WARN] Please either set `enabled = true` in the [anonymous-telemetry] section of pants.toml to enable sending anonymous stats to the Pants project to aid development, or set `enabled = false` to disable it. No telemetry sent for this run. An explicit setting will get rid of this message. See https://www.pantsbuild.org/v2.15/docs/anonymous-telemetry for details.
2.15.0

So everything is a cache hit except for the configure binding which has the argument:

                "--pants-config",
                "/home/jsirois/support/scie-pants/Tobias/two/pants.toml",

That gives it a different configure binding from the one/ directory. After configure runs, the install binding gets a cache hit - it is no different from the install done when pants was run from one/. The issue here is the cache miss, but its fast; so that's not much of a user-facing problem. The user-facing problem is that configure hits the network and that is unexpected when Pants of a given version has already been installed.