cargo2nix

Nixify your Rust projects today with cargo2nix, bringing you reproducible builds and better nix caching.

This repository hosts two components:

A Nixpkgs overlay, located at the /overlay directory, providing utilities to build and test your Cargo workspace.
A utility written in Rust to generate version pins of crate dependencies.

Together, these components will take an existing Cargo.lock and delegate the process of fetching and compiling your dependencies (generated by Cargo) using the deterministic Nix package manager.

Run it now

This project assumes that the Nix package manager is already installed on your machine. Run the command below to get cargo2nix on your $PATH:

nix shell github:cargo2nix/cargo2nix/master

cargo2nix --version

# In directory with Cargo.lock & Cargo.toml files
cargo2nix -f
# Results in Cargo.nix
# See exmaples for how to write a flake that consumes this file for commands
# such as:
# nix develop
# nix build

How to use this for your Rust projects

Build Rust crates purely & reproducibly with all native deps and Rust toolchain provided
Develop with those same dependencies available to your shell
Debug individual dependency builds in a pure environment

As a build system

The basic process of converting an existing Cargo project to cargo2nix boils down to the following steps:

Generate a Cargo.nix file by running cargo2nix -f at the root of your Cargo workspace.
Create a default.nix file which imports Nixpkgs with the cargo2nix and rust-overlay overlays and builds your project using the Cargo.nix file from earlier.
Run nix-build to compile and/or test your project.

Check out our series of example projects which showcase how to use cargo2nix in detail.

Development environment dependencies

In this repo, simply use nix develop and even if you are on a bare NixOS system or fresh OSX environment with no dependencies or toolchains installed, you will have everything you need to run cargo build. See the devShell attribute in flake.nix to see how to prepare this kind of shell.

Declarative debug & development shell

You can load a nix shell for any crate derivation in the dependency tree. The advantage of this shell is that in this environment users can develop their crates and be sure that their crates builds in the same way that cargo2nix overlay will build them.

To do this, first find the .drv for your dependency by using, for example, nix show-derivation | grep cargo

nix show-derivation | rg -o "/nix.*crate.*serde.*drv"
/nix/store/ql6bj0w83bh1ilny0k0iq6d6jq64bjsh-crate-serde-1.0.127.drv

nix develop --ignore-environment /nix/store/ql6bj0w83bh1ilny0k0iq6d6jq64bjsh-crate-serde-1.0.127.drv

# the environment is now as it is when nix builds the package
echo $src 
/nix/store/xci6vnk5fixxgpiwswp8f3vpmc7rwmhk-serde-1.0.127.tar.gz

# If you are working on a dependency and need the source (or a fresh copy) you
# can unpack the $src variable. Through nix stdenv, tar is available in pure 
# shells
mkdir debug
cp $src debug
cd debug
tar -xzfv $(basename $src)
cd <unpacked source>

You will need to override your Cargo.toml and Cargo.lock in this shell, so make sure that you have them backed up if your are directly using your clone of your project instead of unpacking fresh sources like above.

Now you just need to run the $configurePhase and $buildPhase steps in order. You can find additional phases that may exist in overrides by running env | grep Phase

echo $configurePhase 
# runHook preConfigure runHook configureCargo runHook postConfigure

runHook preConfigure
runHook configureCargo
runHook postConfigure

echo $buildPhase
# runHook overrideCargoManifest runHook setBuildEnv runHook runCargo

runHook overrideCargoManifest  # This overrides your .cargo folder, e.g. for setting cross-compilers
runHook setBuildEnv  # This sets up linker flags for the `rustc` invocations
runHook runCargo

If runCargo succeeds, you will have a completed output ready for the (usually) less interesting $installPhase. If there's a problem, inspecting the env or reading the generated Cargo.lock etc should yield clues. If you've unpacked a fresh source and are using the --ignore-environment switch, everything is identical to how the overlay builds the crate, cutting out guess work.

Common issues

When building sys crates, build.rs scripts may themselves attempt to provide native dependencies that could be missing. See the overlay/overrides.nix for patterns of common solutions for fixing up specific deps.

To provide your own override, pass a modified packageOverrides to pkgs.rustBuilder.makePackageSet':

  rustPkgs = pkgs.rustBuilder.makePackageSet' {
    # ... required arguments not shown
  
    # Use the existing all list of overrides and append your override
    packageOverrides = pkgs: pkgs.rustBuilder.overrides.all ++ [
    
      # parentheses disambiguate each makeOverride call as a single list element
      (pkgs.rustBuilder.rustLib.makeOverride {
          name = "fantasy-zlib-sys";
          overrideAttrs = drv: {
            propagatedNativeBuildInputs = drv.propagatedNativeBuildInputs or [ ] ++ [
              pkgs.zlib.dev
            ];
          };
      })
      
    ];
  };

Features in the Cargo.lock may not match what's used in cargo build. You can configure features for each dependency manually without much effort by finding it in the generated code. The root cause is likely to be found in build.rs setting features that don't make it into the lock file.
When re-vendoring rust-overlay or cargo2nix, pay attention to the revs of nixpkgs, the rust-overlay, and the cargo2nix overlay. Certain non-release versions of nixpkgs-mozilla (no longer used) have shipped with a rustc that doesn't include zlib in its runtime dependencies.
Many crates.io public crates may not build using the current Rust compiler, unless a lint cap is put on these crates. For instance, cargo2nix caps all lints to warn by default.
Toml parsing / conversion issues Error: Cannot convert data to TOML (Invalid type <class 'NoneType'>)

jq and remarshal are used to read & modify toml files in some cases. Lines of the form: [key."cfg(foo = \"a\", bar = \"b\"))".path] could produce breakage when jq output was fed back to remarshal. There are workarounds in place to catch many cases. See #149 for more information and report any newly found breakage until a total solution is in place.
Git dependencies and crates from alternative Cargo registries rely on builtins.fetchGit to support fetching from private Git repositories. This means that such dependencies cannot be evaluated with restrict-eval applied.

Also, if your Git dependency is tied to a Git branch, e.g. master, and you would like to force it to update on upstream changes, you should append --option tarball-ttl 0 to your nix-build command.

Design

This Nixpkgs overlay builds your Rust crates and binaries by first pulling the dependencies apart, building them individually as separate Nix derivations and linking them together. This is achieved by passing custom linker flags to the cargo invocations and the underlying rustc and rustdoc invocations.

In addition, this overlay takes cross-compilation into account and build the crates onto the correct host platform configurations with the correct platform-dependent feature flags specified in the Cargo manifests and build-time dependencies.

Credits

The design for the Nix overlay is inspired by the excellent work done by James Kay, which is described here and here. His source is available here. This work would have been impossible without these fantastic write-ups. Special thanks to James Kay!

License

cargo2nix is free and open source software distributed under the terms of the MIT License.

superboum/cargo2nix