Hardcoded base GHC packages

[domenkozar]

Right now stack2nix has GHC 8.0.2 hardcoded in a few places, we should make it configurable.

[kirelagin]

So, asking stack which version of GHC it needs to build the project according to the resolver would be the right thing to do, right?

[kirelagin]

I found only three places:

1. Ensuring GHC is available in Stack2nix.hs
2. Default value in the generated nix expression in Render.hs
3. The list of base packages in Render.hs

The first two are straightforward, as you are inside HasEnvConfig env, you can just query the GHC version.

The third one is somewhat tricky, but I believe there has to be a way to get this list from stack, as it uses it internally. Would querying stack’s “global hints” be a good enough approximation?

[kirelagin]

I believe, it is pretty much what is needed...

› stack query compiler
wanted: ghc-8.0.2
actual: ghc-8.0.2

› stack query global-hints | sort

array: '0.5.1.1'
base: '4.9.1.0'
binary: '0.8.3.0'
bytestring: '0.10.8.1'
containers: '0.5.7.1'
deepseq: '1.4.2.0'
directory: '1.3.0.0'
filepath: '1.4.1.1'
ghc-boot-th: '8.0.2'
ghc-boot: '8.0.2'
ghc-prim: '0.5.0.0'
ghc: '8.0.2'
ghci: '8.0.2'
hoopl: '3.10.2.1'
hpc: '0.6.0.3'
integer-gmp: '1.0.0.1'
pretty: '1.1.3.3'
process: '1.4.3.0'
rts: '1.0'
template-haskell: '2.11.1.0'
time: '1.6.0.1'
transformers: '0.5.2.0'
unix: '2.7.2.1'

(The only difference with the list in Render.hs is that here Cabal, haskeline, terminfo, and xhtml are missing for some reason.)

[domenkozar]

Third one can just be a union of all packages needed. See top of the list here: https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/haskell-modules/configuration-ghc-8.2.x.nix

[domenkozar]

@kirelagin are you planning to work on this?

[kirelagin]

Uhm, yeah, I can do that, but I'm afraid I didn't get your last comment. I think the list in nixpkgs is correct and the one stack outputs is incomplete. Where can we get a complete list of base packages for each version of ghc (other than copying it from trac and maintaining manually)?

[kirelagin]

@domenkozar How about the following plan:

1. Query stack to get the compiler version it wants.
2. Run $(nix-build '<nixpkgs>' -A pkgs.haskell.compiler.ghc<version> -Q --no-out-link)/bin/ghc-pkg --no-user-package-db dump and parse its output to get the list of packages (I think stack has a couple of utility functions to help with that).

[domenkozar]

@kirelagin I think that might be too stateful, but worth a try.

To convert from stack compiler version to nix, use https://hackage.haskell.org/package/stack-1.6.3/docs/Stack-Config-Nix.html#v:nixCompiler

And we can ask @snoyberg how to best parse ghc-pkg dump and map it to a list of core libraries.

[kirelagin]

Yes, I saw this function. Unfortunately the list it uses internally is already outdated, there is 8.2.2 and no 8.2.1 in nixpkgs :(. A better solution would be to push a patch into nixpkgs that would provide attributes called after major-only ghc versions pointing to the latest minors.

I also don’t think I’ll be able to reuse this function directly, as it returns the entire attribute path (haskell.compiler.ghc802), and I will probably need just the last attribute name, e.g. to access haskell.packages.ghc802. But that’s not a big deal, it seems that stack returns the wanted version including the minor component and I’ll be able to filter the dots myself :).

[domenkozar]

This is why I think it's easier to manually dump the lists (maybe write a script) and just maintain these manually in stack2nix. Core package list doesn't depend on minor version, and major GHC release is done very infrequently, enough for us to be able to keep up to date. Ideally nixpkgs would expose this in a better way, I'm all for doing it there.

[snoyberg]

I think you already came to this conclusion, but: if you can, I'd advise against using the global hints. It's much more reliable to query GHC itself.

As to how to query: I know code duplication is a bad thing, but given how unstable the Stack internal APIs still are, it's probably easier to just parse the ghc-pkg output directly. A simple command line to dump the list of global packages in an easy-to-parse format is:

stack exec --no-ghc-package-path -- ghc-pkg list --no-user-package-db --simple

If you're already in a Stack environment, you can achieve this instead with:

env GHC_PACKAGE_PATH=: ghc-pkg list --no-user-package-db --simple

[domenkozar]

@kirelagin first part is done: https://github.com/input-output-hk/stack2nix/pull/87/files

I'll use @snoyberg approach to get base libs next and this should be done.

[kirelagin]

Ok! I’m sorry I couldn’t get to implementing this :(. Looking forward to your fix.

[domenkozar]

I still have some WIP for this somewhere, but it wasn't finished.

[domenkozar]

A better way would be to move the filtering that is now in Render.hs into Stack.hs so it happens before package generation - which should also spare some cycles.