er-nix

Common Nix code for Earnest Research

Table of Contents

er-nix

Installing our development tools

If you are a nix enthusiast / power user please take a look at Nix Home Manager, otherwise read on for a simple setup.

This is a way to share our development tools with all developers via a custom nix channel.

Make sure ~/.nix-profile/bin is in your PATH.

Add this repository as a nix channel:

$ nix-channel --add  https://github.com/EarnestResearch/er-nix/archive/master.tar.gz er-nix

(or verify you have it already with nix-channel --list)

Use it to install our packages, e.g.

$ nix-channel --update
$ nix-env -f '<er-nix>' -i  -A 'pkgs.okta-aws-login'

To keep up with new releases:

$ nix-channel --update
$ nix-env --upgrade

Quick start

`nixpkgs/default.nix`

Put the following in nixpkgs/default.nix:

{ system ? builtins.currentSystem }:

let
  er-nix = import (builtins.fetchGit {
    url = "git@github.com:EarnestResearch/er-nix";
    ref = "refs/heads/master";
    # git ls-remote git@github.com:EarnestResearch/er-nix refs/heads/master | awk '{ print "rev = \""$1"\";" }'
    rev = "127c670a010cb60f3cd6bc89b2622562ae3ba82b";
  });
in
  er-nix.pkgsForSystem(system)

This will provide a patched nixpkgs which includes haskell-nix and some local changes for the current system.

`default.nix`

Import nixpkgs like this:

let
  pkgs = import ./nixpkgs {};
in
  { /* your derivation here */ }

Cross-system builds

The build can be parameterized on system, for instance to support building Docker images from a MacBook using a remote builder. Pass it as an argument to your ./nixpkgs:

{ system ? builtins.currentSystem }:

let
  pkgs = import ./nixpkgs { inherit system; };
in
  { /* your derivation here */ }

The system can then be overridden from the command line:

nix build -L -f . --argstr system x86_64-linux

Building Docker images

If you have a Docker image (built with dockerTools.buildImage, you can push this to AWS ECR with something like:

with (import ./nixpkgs {});
let
  dockerBuild = { /* Docker derivation here */ };
in

callPackage earnestresearch.lib.docker.upload-to-aws {
  dockerArchive = dockerBuild;
  dockerName = "my-service";
  dockerTag = builtins.getEnv "CODEBUILD_RESOLVED_SOURCE_VERSION";
}

One can invoke this (if it's in the file release.nix):

nix-build -L -f ./release.nix

Cabal utilities

Load in your shell.nix

let
  hsPkgs = import ./default.nix { inherit pkgs; };
  cabalProject = pkgs.earnestresearch.lib.cabal.project hsPkgs "your-project-name";

and then add cabal new-run shell wrappers for all executables defined in your project with

  buildInputs = with pkgs; [
  ...other stuff...
  ] ++ cabalProject.projectApps;

Custom packages

er-nix.pkgs provides some additional packages not available in the standard nixpkgs. These may be useful in your own development configurations:

Tools

We also add some development tools in er-nix.tools. One reason they're distinct from packages is that they may require arguments to configure. pkgs.haskell-nix.tool provides a lot, so this is mainly a holding area for things that aren't yet there.

haskell-language-server

Haskell Language Server is a language server implementation that should work in any editor with an LSP client. It must be compiled with the same version of ghc as used by the project. We provide a function here to fetch the haskell-language-server-wrapper and cached haskell-language-server binaries for various Haskell versions. To add it to your environment:

nix-env -if https://github.com/EarnestResearch/er-nix/tarball/master -A tools.haskell-language-servers

home-manager users can add the values to home.packages:

let
  er-nix = import sources.er-nix;
{
  home.packages = [
    # Your other nixpkgs here
  ] ++ builtins.attrValues er-nix.tools.haskell-language-servers;
}

There is also a tools.haskellLanguageServersFor function that takes an array of ghcVersions, but this isn't guaranteed to be in our cachix. If you need more, please consider a PR here.

hopenpgp-tools

hopenpgp-tools is referenced in Earnest's GPG instructions, but currently marked as broken in vanilla nixpkgs. We provide a working derivation here as a convenience. To add it to your environment:

nix-env -if https://github.com/EarnestResearch/er-nix/tarball/master -A tools.hopenpgp-tools

home-manager users can add the values to home.packages:

let
  er-nix = import sources.er-nix;
{
  home.packages = [
    # Your other nixpkgs here
  ] ++ builtins.attrValues er-nix.tools.hopenpgp-tools;
}

Development

Updating sources

We use niv to manage our source pins. niv is available in the nix-shell for this project. To update a single source:

niv update nixpkgs

To update all sources:

niv update

Automated updates

The upgrade.yml workflow runs a niv update and fetches the latest LTS on the second and fourth Tuesday of every month.

It is not uncommon for builds to time out, particularly on ghc-8.10.4. Typically someone has to restart it. Enough forward progress is made in cachix, it usually works the second or third time, depending on how much changed.

To improve cache hits in projects, copy test/cabal.project.freeze into your project or template. There is not currently any automation around this process.