NixOS/nixos-channel-scripts

Revive command-not-found support

edolstra opened this issue · 25 comments

edolstra commented

We no longer generate programs.sqlite because that would require downloading all packages from cache.nixos.org. Need a solution. E.g. Hydra could upload a content listing of every NAR to the cache, which generate-programs-index.pl could download (with some local caching).

domenkozar commented

I'm thinking how command-not-found could also work for custom hydra installations. Maybe we could use multiple outputs to generate a list of installed files and command-not-found would depend on that output for all pkgs?

domenkozar commented

It would need to be pluggable, so maybe that's not ideal as it would have a different hash on hydra vs normal installation that wouldn't generate the lists.

vcunat commented

It would be nice if the hook worked in 16.03, but we should at least put it into release notes if it won't. (funnily enough, we have an item in there improving the hook :)

vcunat commented

I hoped the file list in a NAR would be decodable from a very short prefix of the NAR, but that seems not to be the case.

As you suggest, it currently seems easiest to:

  • Create *.narlist next to *.narinfo when uploading to the cache; perhaps it might include listing of all files, as the extra size should be bearable at this point. (And we might compress this file; it should compress well.)
  • After all is finished and release should happen, download all .narlist and generate file database(s) (at least the one for bin/).
  • Optionally, add a web service querying those databases.
layus commented

While we are designing a new system, it may be a nice bonus to make it general enough to support other use cases.
I am thinking in particular to LaTeX wrappers able to download the right packages for a given document. That would be very useful for per-document nix-shell environments now that we have a fully modular tex distribution.

vcunat commented

In case of TeX the situation is easier due to upstream providing listings of all package contents. ftp://tug.ctan.org/pub/tex/historic/systems/texlive/2015/tlnet-final/tlpkg/texlive.tlpdb.xz

aloiscochard commented

Is there a workaround for this, so it could fail temporarily silently?

The error is very verbose when using fish.

domenkozar commented

I've disabled it for now: NixOS/nixpkgs@2a3c0ca

  • 👍2
chris-martin commented

Thanks, it was getting embarrassing while I'm trying to show nix off 😄

FRidh commented

While we are designing a new system, it may be a nice bonus to make it general enough to support other use cases.

There was also talk of some kind of dependency solver for precompiled binaries. That could also benefit from such database.

chris-martin commented

Would it be absurd to suggest explicitly passing a list of expected bin/ outputs to mkDerivation? It might be nice to have that anyway so the build process can do a sanity check to verify that bin contains the right things.

  • 👍1
domenkozar commented

@edolstra if you implement hydra uploading file contents to S3, I'm willing to take care of the rest.

bjornfor commented

Any progress? I hit this issue almost every day...

  • 👍2
bjornfor commented

My current workaround: use a Ubuntu VM to figure out command -> package-name mapping... Should I laugh or cry? :-)

Myrl commented

@bjornfor I have an outdated VPS just for this.

domenkozar commented

Thanks!!

puffnfresh commented

🎆

  • 🎉1
bjornfor commented

Yay! Thank you, @edolstra!

canndrew commented

So... how do I get this working? If I run command-not-found I get:

DBI connect('dbname=/nix/var/nix/profiles/per-user/root/channels/nixos/programs.sqlite','',...) failed: unable to open database file at /run/current-system/sw/bin/command-not-found line 13.
cannot open database `/nix/var/nix/profiles/per-user/root/channels/nixos/programs.sqlite' at /run/current-system/sw/bin/command-not-found line 13.

Is there a way to fix it on an existing NixOS install?

vcunat commented

@canndrew: apparently your channel isn't new enough; try sudo nix-channel --update.

canndrew commented

Nope, that didn't change anything. I have nix.nixPath set to point at my own local clone of nixpkgs but that is currently up-to-date with nixpkgs-channels/nixos-unstable. Also I only installed NixOS a couple of weeks ago, after this bug was closed.

Any more ideas?

chris-martin commented

I also haven't seen this working yet. I don't use nix-channel at all anymore. Is that a requirement?

edolstra commented

Yeah, you have to use nix-channel because command-not-found looks specifically in /nix/var/nix/profiles/per-user/root/channels/nixos/programs.sqlite.

  • 👍2
domenkozar commented

Note that generation was disabled again, please comment on #9