matthewbauer/ghc-nix

Awesome! I've got questions!

angerman opened this issue · 7 comments

angerman commented

This looks really neat. Here's a few questions I have though:

  • do you have timing information for identical rebuilds?
    • first run: (everything needs to be built) cabal vs. ghc-nix. I guess you could force ghc-nix to rebuild everything by giving the hs-builder an argument and embed that into the derivation, e.g. a splace or some # comment, to prevent pre-existing cache-hits.
  • second run: (evertying is already built) cabal vs. ghc-nix
  • we essentially generate a nix expression for each haskell file, this seems pretty similar to snack, except that we don't try to do it all in nix. Does the explosion of nix-expressions cause issues?
  • any reason this would not work with Setup.hs?
  • Finally, ghc-nix calles out to nix a lot, unless I missread the source, so this won't work inside a nix-build unless we have recursive nix?

Thanks! Amazing work!

ocharles commented

Do you have timing information for identical rebuilds?

How's this?

Build Time
fused-effects-1.0.0.0 (after nix-collect-garbage) 23s
fused-effects-1.0.0.0 (after cabal clean, no ghc-nix) 10.96s
fused-effects-1.0.0.0 (ghc-nix with 100% cache hits) 15s

I'm going to do some quick work on profiling to get better timings for how long it takes to call nix-instantiate, nix-build and nix make-content-addressable. I wouldn't say any of these timings are too interesting right now, I basically vomited out the simplest thing that could possibly work!

we essentially generate a nix expression for each haskell file, this seems pretty similar to snack, except that we don't try to do it all in nix.

Yes, it's similar, but the point of this to drive the build in ghc-nix and use nix make-content-addressable to get early cut off. Snack cannot do this, so that's what is new here.

Does the explosion of nix-expressions cause issues?

I dunno 😄 I don't have a good answer for that.

any reason this would not work with Setup.hs?

It does, doesn't it? cabal build is the same thing, isn't it? I would imagine ./Setup -w ghc-nix should work, assuming ./Setup gives you a -w option.

Finally, ghc-nix calles out to nix a lot, unless I missread the source, so this won't work inside a nix-build unless we have recursive nix?

Yes, this is exactly the idea. The end goal is to have this as something you can drop in to haskellPackages (or haskell.nix) to do all Haskell builds. At that point, you need recursive Nix.

Let me know if that helps, and if you have any other questions!

angerman commented

Am I reading those timing information right? ghc-nix currently takes 100% (uncached) and 50% (cached) more time thanghc --make?

I'd eventually like to try this out with haskell.nix once we have recursive nix proper in nix. Especially if we can somehow get better performance.

ocharles commented

Looks like it, yep. But I'm not too bothered by the numbers yet. I'm sure it will get faster than this! For one, I'm not even certain if my whole 'fork a GHC for each file is necessary'. Maybe we can act more like GHC --make with just one process

  • 👍1
angerman commented

I'm absolutely looking forward to it!

expipiplus1 commented

Looks like it, yep. But I'm not too bothered by the numbers yet. I'm sure it will get faster than this! For one, I'm not even certain if my whole 'fork a GHC for each file is necessary'. Maybe we can act more like GHC --make with just one process

Would this require some modifications to GHC itself to have it call nix-build internally?

ocharles commented

That's certainly one option, but unlikely to happen. I think I was more thinking about having ghc-nix be smarter, but ultimately I can't really remember what I had in mind.

matthewbauer commented

I've done some work improving performance. Here's the updated times:

Build Time
fused-effects-1.1.1.2 (after nix-collect-garbage) 10.066s
fused-effects-1.1.1.2 (after cabal clean, no ghc-nix) 10.230s
fused-effects-1.1.1.2 (ghc-nix with 100% cache hits) 2.244s

I'm using:

nix develop github:matthewbauer/fused-effects/add-flake -c hyperfine -L compiler ghc-nix -w 1 -p "./nix-clean.sh && cabal clean && cabal configure -w {compiler} -j" "cabal build -w {compiler}"

to time 1 and

nix develop github:matthewbauer/fused-effects/add-flake -c hyperfine -L compiler ghc,ghc-nix -w 1 -p "cabal clean && cabal configure -w {compiler} -j" "cabal build -w {compiler}"

to time 2 & 3.

out of this branch: https://github.com/matthewbauer/fused-effects/tree/add-flake

I'm not sure why ghc-nix is slightly faster than cabal without cache... Might be that in nix max-jobs = 10 for me and in ghc jobs = $ncpus. I'm not sure how ncpus is calculated.

  • 👀1