deislabs/bindle

Bindle variants

npmccallum opened this issue · 3 comments

Let's say that there is some upstream source code. It has multiple build options (for example, cargo features). We now want to produce builds for the Cartesian product of these features. In bindle, these would all have to have separate names and the correlation between the source name and the build variant name is either lost or implicit or encoded in a non-obvious way.

Combined with #266, naming for binaries produced from an upstream is really a 4-tuple:

  • source name
  • build variant
  • source version
  • build version

One could just say that bindle's scheme represents the build variant and version and be done. However, the reality is that consumers of these build artifacts almost always want to refer to the upstream source name and version and only consider the build variant and version secondarily. So while this approach optimizes for Bindle's concern it doesn't optimize for the user concern.

In Bindle, you would not produce separate packages for each variant. here's an example.

The way WasmCloud handled this was to store all build variants in one Bindle and use groups to determine which one to fetch. And that was basically the intended usage of Bindle. /cc @thomastaylor312 (who wrote WasmCloud's support).

For example, WasmCloud distributes Linux, Windows, and macOS binaries of their providers. And I believe they also distribute ARM and AMD64 versions of each. They reference all six parcels on one invoice, organized into six groups (windows-amd64, windows-arm64, linux-amd64...). Then, on pulling, the client indicates which group it wants to pull, and fetches only those packages.

Bottom line: A bindle is not a tarball where you just fetch the entire thing and dump it onto the filesystem. The idea of the tree-like structure of a bindle is that you can reason about its contents, and select only the things you want out of the package.

Yeah, @technosophos is correct here. This is the example for a wasmCloud provider, which contains multiple binaries of different targets: https://wasmcloud.dev/reference/bindle/provider-bindles/#a-full-example. The host runtime then selects the proper things it needs from the bindle depending on the platform it is running on

In general the idea is for someone constructing a bindle to be able to express arbitrary connections between things within a bindle. Particularly with webassembly, these could be binary artifacts from various different projects.