idea: fetch build system specific dependency metadata

[andir]

For Rust and Go (and other languages) it is common to either have a big IFD derivation as part of the nix build or to have a 2nix tool generate a ton of (garbage) code. Especially in nixpkgs or pure-eval mode that is a big deal as you can't read the lock files within the repositories that you are trying to package.

For at least Rust (and others, with some effort) we could add additional source information in the lockfile.

I'll try to illustrate this with a Rust project called ripgrep.

When you run npins add github burntsushi ripgrep npins will track the latest release of the repository, fetch tarball and store the hash in the lockfile. When you package ripgrep with Nix you need to provide the crates ripgrep depends on. As describe above that is usually IFD or some convoluted code generation step. What npins could be doing in this case is parsing the Cargo.lock toml and generate additional metadata as part of the lockfile.

We should add a --extra-metadata rust,npm,.... switch to the add command.

This would be the same that import-cargo (https://github.com/edolstra/import-cargo/blob/25d40be4a73d40a2572e0cc233b83253554f06c5/flake.nix#L8-L68) or what buildRustPackage with cargoLock.lockFile = ... in Nixpkgs (https://github.com/NixOS/nixpkgs/blob/ce1f9354959ae1493916f2e551ecc32e79b4a473/pkgs/build-support/rust/import-cargo-lock.nix) do when they can access the Cargo.lock file.

In the lockfile it could look like this (pseudo code):

{
   "ripgrep": {
       "type": "github",
       /* common attributes */
      "extraAttributes": {
         "cargoDependencies": {
             "atty-0.2.14": /* we could drop the name and return a list, this example name matches with nixpkgs would pick right now */ {
                /* all the values "raw" as they are in the Cargo.lock, we don't want to remove data or mangle it */
                "name": "atty",
                "version": "0.2.14",
                "source": "registry+https://github.com/rust-lang/crates.io-index",
                "checksum": "d9b39be18770d11421cdb1b9947a45dd3f37e93092cbf377614828a319d5fee8",
                "dependencies":  ["hermit-abi", "libc", "winapi", ]
         }
     }
   }
}

In our Nix code that parses the lockfile we could then attach those as additional (passthru-style) parameters to the output path just like we already provide version, revision etc...

This should be rather simple to add as we are already downloading the sources. We then have to read the Cargo.lock file (in the root of the repository, for now) and read the metadata.