opam2nix is experimental software

I'm hoping to make it stable and a future part of nixpkgs. But for now, it's just this code which might work sometimes, and will probably change a lot. It's quite complex, and it's likely that some simpler patterns will make themselves obvious after some more extensive use.

Important note on using in your own project:

Don't try to clone opam2nix as part of your own derivation. If you instead copy the current nix/release.nix into your own source code you can import just that one file (unsing `pkgs.callPackage) and it'll in turn clone the relevant commit from this repository and bootstrap itself. If needed, you can replace in the git URLs or revisions with the latest commits (or a commit in your fork).

Getting started:

The easiest way to get started is to check out the examples/ directory. It's got small, working examples that you can probably adapt to your own use very easily.

Detailed usage instructions:

One you've copied release.nix as opam2nix-packages.nix, you can use it like so:

let
  opam2nix = pkgs.callPackage ./opam2nix-packages.nix {};
in

# for simply building one package, use:
opam2nix.buildPackage "someOpamPackage";

# for non-opam software, you'll build selections based on direct dependencies,
# and include each direct dependency in your `buildInputs`. This will
# include the `ocaml` dependency:
mkDerivation {
  buildInputs = opam2nix.build { packages = ["lwt"]; };
  ( ... )
};

# If you are developing your own package with an .opam file, you can save yourself the
# trouble of replicating your dependencies in `nix`-land by using the `buildOpamPackage` function
# instead of `mkDerivation`:
opam2nix.buildOpamPackage rec {
  name = "pkgname-version";
  src = ( ... );
};

The utility buildPackageSet is very useful for re-exposing all transitive ocaml dependencies for debugging purposes:

passThru.ocamlPackages = opam2nix.buildPackageSet { packages = ["lwt"]' };

This can be used with e.g. nix-build -A ocamlPackages.lwt default.nix if you need to build an individual dependency (but in your project's configuration; i.e. taking all optional dependencies and constraints into account). It accepts all the same arguments as build and produces an object with keys for every transitive dependency, rather than a list of direct dependencies.

The build, buildPackageSet and buildOpamPackage functions all accept the union of options accepted by the lower level selectionsFile and importSelectionsFile functions (see "Configuration" section).

Configuration

opam2nix.selectionsFile takes an attribute set with the following properties:
- ocamlAttr: defaults to "ocaml"
- ocamlVersion: default is extracted from the derivaiton name of pkgs.<ocamlAttr>, should rarely need to be overriden
- basePackages: defaults to ["base-unix" "base-bigarray" "base-threads"], which is hacky.
- packages: string list of package names
- args: extra list of string arguments to pass to the opam2nix tool (default [])
- extraRepos: extra list of string arguments to pass to the opam2nix tool (default [])
opam2nix.importSelectionsFile selections_file takes an attribute set with the following properties, all optional:
- pkgs: defaults to the pkgs set opam2nix was imported with
- overrides: function accepting a world argument and returning attributes to be overriden / added
- opam2nix
- extraPackages: extra package definitions (obtained by importing the result of opam2nix generate. Automatically generated by running opam2nix generate on each item in extraRepos by default.
- extraRepos: convenient shortcut for populating extraPackages, accepts the same values as the extraRepos property passed to selectionsFile
opam2nix.build, opam2nix.buildPackageSet:
- accepts any option accepted by either selectionsFile or importSelectionsFile
opam2nix.buildOpamPackage:
- accepts any option accepted by either selectionsFile or importSelectionsFile, except packages

Hacking

This repo contains generated .nix expressions, as well as some overrides required for a bunch of packages which don't quite work out of the box.

To add specific package versions, add them in packages.repo and rebuild.

Manual operation / how does it all work?

You hopefully don't have to know in order to use this repo - the above instructions should be enough to use these packages without ever delving into the command line utility yourself, but here's how it works:

Step 1: generate a set of `nix` package definitions based on an opam repository.

$ opam2nix repo --src ~/.opam/repo/ocaml.org --dest <dest>/nix/packages --cache <dest>/cache '*@latest'

This traverses the repo, scans the packages you've selected, downloads sources that it hasn't cached, reads opam files for dependencies, and spits out a .nix file for each version of each package.

Step 2: Implement manual overrides

The above step generates "pure" package definitions based only on the information in the opam repository. But in order to integrate cleanly with nixpkgs, some generated packages need to be modified. This is implemented as a nix expression which wraps the generated packages. You should probably start with the repo/default.nix and repo/overrides from the opam2nix-packages repo, and make any changes you need from there.

Step 3: select exact versions of each dependency

The generated .nix files are pretty dumb - they know the difference between mandatory and optional dependencies, but that's about all. They rely on you giving them a valid set of dependencies which satisfy all versioning constraints, conflicts, etc. Conveniently, this is exactly what opam's solver does - but instead of actually installing everything, let's just get it to create a nix expression of the packages it would install:

$ opam2nix select \
  --repo <dest>/nix \
  --dest <dest>/selection.nix
  --ocaml-version 4.01.0 \
  --ocaml-attr ocaml \
  --base-packages 'base-unix,base-bigarray,base-threads' \
  lwt

You shouldn't modify this selections.nix file directly, as you'll regenerate it whenever your dependencies change. Instead, you should call it from your main .nix file like so:

{ pkgs ? import <nixpkgs> {}}:
let
  selection = pkgs.callPackage ./dest/selection.nix {
    # one day, both of these may be rolled into `nixpkgs`, making them optional:
    opam2nix = pkgs.callPackage /path/to/opam2nix/default.nix {};
    opamPackages = import ./<dest>/nix { inherit pkgs; };
  };
in
{
  name = "foo-bar";
  buildInputs = [ selection.lwt ];
  # ...
}

vklquevs/opam2nix-packages