Build native-compiles every package every time my emacs config changes.

Hi!

I use nix-doom-emacs like this, following the suggestion in #60 (comment) for tangling my literate config (here).

  programs.doom-emacs = {
    enable = true;
    doomPrivateDir = (import ./doom.d) {
      inherit lib;
      inherit (pkgs) stdenv emacs coreutils;
    };
  };

Every time I change my Emacs configuration at all and rebuild, it takes a long time while Nix recompiles every Emacs package. I have the nix-community cachix added, but it doesn't appear to get used at all. Not sure why.

I want to figure out the problem with cachix but in the meantime, I noticed that this rebuild happens even if the set of packages changed. I played around with it, and turns out it's pretty easy to cache the straight-env by only including init.el and packages.el: znewman01@a076453

Would something like that be useful to commit back? It feels strictly better to me than rebuilding each time. Maybe a flag to turn off the behavior in case someone has an interesting package installation setup that relies on some other files?

For reference: my flake.nix (pointing at my fork of nix-doom-emacs) and flake.lock.

It's pretty neat.

@thiagokokada:

I feel like we thought about this at one point already, maybe it just got lost in a thread? Feels like a bit of a hack but I'm up for a PR with a flag to opt out I think.

Yeah, the idea is pretty neat, however there are a few issues that I can think of.

AFAIK, there is nothing in doom-emacs that prohibits you to declare packages in places that are not the package.el file. Also, you can create custom modules that also declare packages. So in both of those cases this idea from @znewman01 would fail to take the new packages and trigger a new build.

We probably need a way to declare the files that will trigger the rebuild, so the user can customize themselves.

Yeah, you're both right—it's definitely a hack, which is why I wanted to ask before just sending a PR in. 🙂

WDYT about something like:

a new doomPackageFilesDir argument:

{
  # [...]
   /* A Doom configuration directory from which to build the Emacs package environment.

      Can be used, for instance, to prevent rebuilding the Emacs environment
      each time the `config.el` changes.

      Can be provided as a directory or derivation. If not given, package
      environment is built against `doomPrivateDir`.

      Example:
        doomPackageDir = pkgs.linkFarm "my-doom-packages" [
          # straight needs a (possibly empty) `config.el` file to build
          { name = "config.el"; path = pkgs.emptyFile; }
          { name = "init.el"; path = ./doom.d/init.el; }
          { name = "packages.el"; path = pkgs.writeText "(package! inheritenv)"; }
          { name = "modules"; path = ./my-doom-module; }
        ];
   */
   doomPackageDir ? "",
   # ...

That feels okay to me for a first draft—doesn't make any changes to the default behavior, and lets folks opt-in to include files they want. Might be nice later on to add some sugar:

A shortcut for common patterns (e.g., doomPackageDir = [ "init.el" "packages.el" "mod/" ] for filtering doomPrivateDir`.
Automatically creating the empty config.el if needed.
A special case in nix-doom-emacs for the ("just take init.el and packages.el case, which I imagine ). This could be an exported function to apply to ./doom.d.

If something like the above sounds good, let me know!

doomPackageDir seems like a nice idea. Though default should be doomPrivateDir or null, not "". The third bullet point seems not-so-trivial to implement over all supported platforms so don't block on that for the PR. List sounds nice and auto config.el should probably come with a little warning.

If something like the above sounds good, let me know!

Sounds good to me. I wouldn't care to much about about adding the sugar, at least for a first PR.

Also concur that the default should be null and not "".

Just wondering how I would go about enabling this? Can't find docs. Thanks for any help.
Is the idea that we now set doomPackageDir to a folder containing only init.el and packages.el

Docs are only inline right now:

nix-doom-emacs/default.nix

Lines 3 to 23 in c852431

    
             /* A Doom configuration directory from which to build the Emacs package environment. 
        
                Can be used, for instance, to prevent rebuilding the Emacs environment 
        
                each time the `config.el` changes. 
        
                Can be provided as a directory or derivation. If not given, package 
        
                environment is built against `doomPrivateDir`. 
        
                Example: 
        
                  doomPackageDir = pkgs.linkFarm "my-doom-packages" [ 
        
                    # straight needs a (possibly empty) `config.el` file to build 
        
                    { name = "config.el"; path = pkgs.emptyFile; } 
        
                    { name = "init.el"; path = ./doom.d/init.el; } 
        
                    { 
        
                      name = "packages.el"; 
        
                      path = pkgs.writeText "packages.el" "(package! inheritenv)"; 
        
                    } 
        
                    { name = "modules"; path = ./my-doom-module; } 
        
                  ]; 
        
              */ 
        
           ,  doomPackageDir ? doomPrivateDir

(#331 will probably document this somewhere better)

For an example, see how I'm using it: https://github.com/znewman01/dotfiles/blob/c1b35cdfb4ca15a22ab55587fac1116f6a87475a/emacs/default.nix#L6-L27

(#331 will probably document this somewhere better)

If I search for doomPackageDir in this repository, I can't actually find any documentation on it in the repository files, except for what exists in a comment in default.nix. And that documentation doesn't explain which files to put in ./doom.d/ and which ones in ./my-doom-packages/ or ./my-doom-module/.

	/* A Doom configuration directory from which to build the Emacs package environment.

	Can be used, for instance, to prevent rebuilding the Emacs environment
	each time the `config.el` changes.

	Can be provided as a directory or derivation. If not given, package
	environment is built against `doomPrivateDir`.

	Example:
	doomPackageDir = pkgs.linkFarm "my-doom-packages" [
	# straight needs a (possibly empty) `config.el` file to build
	{ name = "config.el"; path = pkgs.emptyFile; }
	{ name = "init.el"; path = ./doom.d/init.el; }
	{
	name = "packages.el";
	path = pkgs.writeText "packages.el" "(package! inheritenv)";
	}
	{ name = "modules"; path = ./my-doom-module; }
	];
	*/
	, doomPackageDir ? doomPrivateDir