Incorrect Dart executable path resolution in sandboxed environments like Nix

Description

The logic for locating the dart executable within webdev and build_daemon makes assumptions about the Dart SDK directory structure that do not hold true when running as a compiled executable in a sandboxed environment like Nix.

The current implementation determines the path to the dart executable by traversing up from the path of the currently running executable (Platform.resolvedExecutable). This assumes a standard SDK layout where the application's executable is in a bin directory two levels below the SDK root. This assumption fails in environments like Nix, where the directory structure is different, leading to an incorrect path for the dart binary.

Context: How jaspr uses webdev

The jaspr build command utilizes the webdev package to compile its web assets. Specifically, it calls webdev's connectClient function to start and communicate with the build_runner daemon. It is during this process that webdev (and its dependency, build_daemon) attempts to locate the Dart SDK to start the daemon process, triggering the bug described above. The failure to correctly locate the dart executable prevents the build daemon from starting, which halts the entire jaspr build process.

Problematic Code

The following code contains the flawed logic:

webdev/webdev/lib/src/util.dart

Line 29 in 186bfe7

/// The path to the root directory of the SDK.

Debugging Output

The following log output from within a Nix build environment clearly demonstrates the issue:

1 --- Debugging dart path resolution ---
2 Platform.resolvedExecutable: /nix/store/gx9i2rvqb3mjlhb2wxznr1y4ip42qaz9-jaspr-v0.21.3-dev/bin/jaspr
3 p.dirname(p.dirname(Platform.resolvedExecutable)): /nix/store/gx9i2rvqb3mjlhb2wxznr1y4ip42qaz9-jaspr-v0.21.3-dev
4 Checking for version file at: /nix/store/gx9i2rvqb3mjlhb2wxznr1y4ip42qaz9-jaspr-v0.21.3-dev/version
5 Version file exists: false
6 _sdkDir: /nix/store/gx9i2rvqb3mjlhb2wxznr1y4ip42qaz9-jaspr-v0.21.3-dev
7 Constructed dartPath: /nix/store/gx9i2rvqb3mjlhb2wxznr1y4ip42qaz9-jaspr-v0.21.3-dev/bin/dart
8 --- End of debugging dart path resolution ---

This output shows that:

Platform.resolvedExecutable correctly points to the compiled application binary.
The code incorrectly assumes the Dart SDK root is two directories above this binary.
The check for a version file at this location fails, confirming the incorrect SDK path.
The final dartPath is constructed within this incorrect directory, resulting in a path that does not exist and causing a
ProcessException: No such file or directory.

Suggested Solution

The mechanism for locating the dart executable should be made more robust. Instead of relying on a fragile directory structure relative to the running executable, the process should simply execute dart and depend on the PATH environment variable to resolve the correct binary. This is a more portable approach that will work correctly across standard installations, Docker containers, and Nix environments.

The code responsible for starting the daemon process should be modified to use 'dart' as the executable, rather than constructing an absolute path based on these flawed assumptions.

Alternatively, let the caller override the dart executable path.

Recently, in order to support the new changes to dart pub global activate, @biggs0125 modified the way we calculate the Dart SDK path to do what your first suggestion recommends:

webdev/webdev/lib/src/util.dart

Line 33 in 2517aa9

final String dartExecutable = Platform.isWindows

. Can you override the webdev dependency path in jaspr to see if that works for you before we publish webdev?

Beyond that, a lot of the downstream tooling depends on the path structure of the Dart SDK, so even if we were to allow overriding the paths we use in webdev, things can break further down if the executable is in an unexpected location. Can you share a bit more on how Dart is installed in a Nix environment?

Thank you. I've tested the changes and it seems to fix the problem above.

The new problem is during the build:

#      > Running command: dart run build_runner build --no-pub --release --verbose --delete-conflicting-outputs --output build/jaspr --define=jaspr_web_compilers:entrypoint=compiler=dart2js --define=jaspr_web_compilers:entrypoint=dart2js_args=["-Djaspr.flags.release=true","-O2","-DPOCKETBASE_URL=http://localhost:8080","-DAPP_VERSION=0.1.0","-DCOMMIT_HASH=d4fe63a-dirty","-DIS_WASM=0"]
#      > [BUILDER] [ERROR] Got socket error trying to find package lints at https://pub.dev.

Even though I pass --no-pub a subsequent dart call seems to not get the arg forwarded. I've not yet found where exactly this happens. I suspect this is in jaspr_web_compilers or in build_runner. I think this should be a new issue though.

Here's a rundown of how Dart works in Nix. It's quite lengthy, but I feel like it's the minimum to get a high level overview of how it works:

How Dart Works in Nix

Understanding how Dart packages are managed in Nix is crucial for debugging and extending the build system.

A. In a users System (Development Environment)

On a user's system, Dart is made available by creating a shell with dart in PATH:

nix-shell -p dart # creates a temporary shell with dart in PATH

Alternatively it can be declaratively installed via a NixOS configuration. There are other ways but they are irrelevant for this guide.

Directory structure:

/nix/store/abc123-dart-3.9.2/
├── bin/
│   ├── dart
│   ├── dartaotruntime
│   └── ...
├── pkg/
│   ├── analysis_server/
│   ├── analyzer/
│   └── ...
└── version

PATH resolution: which dart → /nix/store/abc123-dart-3.9.2/bin/dart

Key characteristics:

Standard SDK structure is preserved
Multiple versions can coexist
PATH-based resolution works seamlessly

In a user's build environment, stateful calls like dart pub get are working normally.

B. In a build Environment for building Nix Derivations

In Nix builds, we don't run dart pub get because:

No network access in sandbox
Non-reproducible (package versions could change)
Dependencies should be declared explicitly

Dependency Management

Instead of pub get, we:

Pre-fetch each dependency into its own Nix store path:

/nix/store/dep1hash-pub-build_runner-2.6.0/
├── lib/
├── pubspec.yaml
└── ...

/nix/store/dep2hash-pub-webdev-3.5.1/
├── lib/
├── pubspec.yaml
└── ...

We read the pubspec.lock file to get the dependency names and versions.

Construct package_config.json pointing to these paths:

{
  "configVersion": 2,
  "packages": [
    {
      "name": "build_runner",
      "rootUri": "file:///nix/store/dep1hash-pub-build_runner-2.6.0/",
      "packageUri": "lib/",
      "languageVersion": "2.12"
    },
    {
      "name": "webdev",
      "rootUri": "file:///nix/store/dep2hash-pub-webdev-3.5.1/",
      "packageUri": "lib/",
      "languageVersion": "2.12"
    }
  ],
  "generated": "2025-01-16T12:00:00.000Z",
  "generator": "nix"
}

Place it in .dart_tool/ before compilation

Build Process Flow

┌─────────────────────────────────────────────────────────────────┐
│ 1. Unpack Phase                                                 │
│    Source → /build/source/                                      │
└─────────────────────────────────────────────────────────────────┘
                              ↓
┌─────────────────────────────────────────────────────────────────┐
│ 2. Configure Phase (dartConfigHook)                             │
│    - Dart SDK in PATH: /nix/store/abc-dart-3.9.2/bin/dart       │
│    - Copy pre-built package_config.json to .dart_tool/          │
│    - All dependencies already in /nix/store/                    │
│    - NO pub get needed!                                         │
└─────────────────────────────────────────────────────────────────┘
                              ↓
┌─────────────────────────────────────────────────────────────────┐
│ 3. Build Phase (dartBuildHook)                                  │
│    - dart compile exe/js/aot-snapshot/etc.                      │
│    - Tools (build_runner, webdev) spawn 'dart' via PATH         │
│    - Read dependencies from .dart_tool/package_config.json      │
└─────────────────────────────────────────────────────────────────┘
                              ↓
┌─────────────────────────────────────────────────────────────────┐
│ 4. Install Phase (dartInstallHook)                              │
│    - Binary → /nix/store/xyz-my-app-1.0.0/bin/                  │
│    - Places the compiled app in it's /nix/store/ path           │
└─────────────────────────────────────────────────────────────────┘

Post-Build Structure

Compiled application:

/nix/store/xyz789-my-app-1.0.0/
└── bin/
    └── my-app  (compiled executable)

Important: Platform.resolvedExecutable in this app points to /nix/store/xyz789-my-app-1.0.0/bin/my-app, which is NOT an SDK directory.

Why PATH-Based Resolution is Critical

Tools like webdev and build_daemon need to spawn dart processes. They should:

Use dart command directly (not construct absolute paths)
Rely on PATH to find the SDK
Inherit environment from parent process

This works because:

Build time: Dart SDK is in PATH with proper structure
Runtime: Applications that need dart declare it as a dependency

The --no-pub Flag Expectation

When we run:

dart run build_runner build --no-pub

All subsequent Dart invocations should respect offline mode, including:

Processes spawned by build_runner or webdev
The build daemon
Code generators

With PATH-based resolution, spawned dart processes:

Use the same Dart SDK as the parent
Inherit environment variables
Work with our pre-constructed package_config.json
Don't attempt network access

TL;DR: Some parts of the Dart/Flutter ecosystem seem to require network access even though --no-pub is used and that is problematic for sandboxed environments like Nix.

Thanks for the detailed walkthrough @fusion44, that was really helpful to get a better understanding as to what's going on.

Just to make sure I'm understanding correctly:

jaspr is directly depending on some internals of webdev that rely on spawning a dart process or looking for specific Dart SDK artifacts.
jaspr is being compiled as an executable, so Platform.resolvedExecutable is the name of the executable, not dart or dartaotruntime, which makes the SDK directory resolution give back a garbage path.

Does that sound right to you?

Sorry for the late answer @bkonyi. Point 1 sounds about right.

For point 2 I'd add that webdev assumed that the dart executable is relative to the executable it was compiled into. This seems to be fixed in main.

No worries!

package:webdev was never actually meant to be consumed directly as an imported package, only as a pub global executable, so we were able to make some assumptions about how it was being run and where Platform.resolvedExecutable points to.

Technically, jaspr is violating package privacy by referencing non-public APIs of package:webdev, which is something we're going to have to work with the maintainers of jaspr to fix.

Also, the logic here that fixes the issue on main is technically incorrect as it breaks if dart isn't explicitly on your path or if you have multiple versions of Dart installed. However, given we plan to rework how webdev is shipped in the near future, this probably won't be a big deal and won't be reverted.

I think this issue is resolved, so I'm going to close it, but if that's not the case please let me know!

@bkonyi yes, that's fine for me as the original problem I've opened this issue for is actually fixed in main.

When you tackle the other issues I've mentioned you can ping me and I'll test them in my Nix env.