Default Nix installations don't include user channels in NIX_PATH, causing inconsistent tool behavior

Question

Default Nix installations don't include user channels in NIX_PATH, causing inconsistent tool behavior

grahamc opened this issue 6 years ago · 31 comments

Should Nix installations include $HOME/.nix-defexpr/channels in the NIX_PATH? Should it appear before or after root channels?

nixpkgs=/nix/var/nix/profiles/per-user/root/channels/nixos/nixpkgs:nixos-config=/etc/nixos/configuration.nix:/nix/var/nix/profiles/per-user/root/channels

$ nix-channel --list             
nixpkgsunstable https://nixos.org/channels/nixpkgs-unstable

$ nix-env -iA nixpkgsunstable.git
installing 'git-minimal-2.16.2'

$ nix-shell -p nixpkgsunstable.git
error: undefined variable 'nixpkgsunstable' at (string):1:94
(use '--show-trace' to show detailed location information)

$ nix-shell '<nixpkgsunstable>' -p git
error: file 'nixpkgsunstable' was not found in the Nix search path (add it using $NIX_PATH or -I), at (string):1:94
(use '--show-trace' to show detailed location information)

$ nix-build '<nixpkgsunstable>' -A git
error: file 'nixpkgsunstable' was not found in the Nix search path (add it using $NIX_PATH or -I)

vs:

$ export NIX_PATH=$NIX_PATH:$HOME/.nix-defexpr/channels

$ nix-env -iA nixpkgsunstable.git    
replacing old 'git-minimal-2.16.2'
installing 'git-minimal-2.16.2'

$ nix-shell -p nixpkgsunstable.git    
error: undefined variable 'nixpkgsunstable' at (string):1:94
(use '--show-trace' to show detailed location information)

$ nix-shell '<nixpkgsunstable>' -p git

[nix-shell:~]$ exit

$ nix-build '<nixpkgsunstable>' -A git
/nix/store/jaani1kx4a4kvz11d0myfkjjiiklfl7w-git-2.16.2

With this configuration, and the explicitly specified nixpkgs, is it possible for users to override nixpkgs without redefining the NIX_PATH? Does it matter?

Answer 1 · 2018-04-01T13:44:56.000Z

Yes, before root IMO.

Answer 2 · 2018-04-01T15:07:12.000Z

Do you know why nix-env and nix-shell are not using the same lookup method?

For everything to be perfectly aligned, nix-channel --list should probably report all the channels on the NIX_PATH instead of using that fixed location.

Answer 3 · 2018-04-01T15:09:53.000Z

@zimbatm Mostly because nix-env is old 😀 It predates nix path.

Answer 4 · 2018-04-01T16:01:49.000Z

I guess it's easier to fix the installer than nix-env :)

Answer 5 · 2018-04-01T16:06:22.000Z

@grahamc Assigned to you, you pretty much own the install process these days 😀 Feel free to make a call on your own or call in @NixOS/nix-core as you see fit.

Answer 6 · 2018-04-01T16:44:13.000Z

I'm happy to implement these changes to the installer and what-not, but'd love input from @NixOS/nix-core too

Answer 7 · 2018-04-01T16:53:12.000Z

This isn't specific to the installer, it's this way to match the defaults from nixos. Also what is the plan for channels/user profiles with the new ui?

Answer 8 · 2018-04-01T17:17:15.000Z

I think we should also update NixOS to have user channels in the NIX_PATH for 18.03 and unstable, is that too big of a change to backport? (cc @vcunat, @fpletz)

Answer 9 · 2018-04-02T15:23:11.000Z

This kind of change seems good to me, if we do it before the official announcement. That would mean to do the change rather quickly now (and test it does what we want), within a couple days.

Answer 10 · 2018-04-03T07:13:57.000Z

@shlevy

Yes, before root IMO.

What is the motivation? Note the Nix docs on NIX_PATH don't explicitly mention what happens in case an entry is mentioned twice. Assuming one can override an entry (I've never tried), then administrators may prefer not to put user channels before root channels.

Answer 11 · 2018-04-03T11:03:57.000Z

@FRidh Earlier paths take precedence, and in general letting users override system-wide policy is the right call (outside of access control issues, of course)

Answer 12 · 2018-04-03T11:18:33.000Z

Previously I believe I've seen Nix complain about multiple entiries-- I think when prepending "~/.nix-defexpr/channels" to NIX_PATH. Not sure if still happens, but just a heads-up.

Answer 13 · 2018-04-03T12:20:54.000Z

Yes, if in my user profile add nixos and my root profile has nixos it will warn, but work properly, on nix-env:

warning: name collision in input Nix expressions, skipping '/home/grahamc/.nix-defexpr/channels_root/nixos'

Answer 14 · 2018-07-23T00:38:41.000Z

It seemed pretty easy to get into a position of being warned and a bit confusing to get out of it - wish I understood the purpose of adding channels_root to ~/.nix-defexpr/.

On my mac running 2.0.4, I ended up just removing all root channels to avoid the duplication - sudo -i su root and nix-channel --remove <name> worked even tho nix-channel --list was not working for root (I guess related to #1548)

Answer 15 · 2018-12-16T14:27:59.000Z

It would be great to have $HOME/.nix-defexpr/channels in NIX_PATH by default. The new Home Manager installation instructions I put together relies on nix-channel --add https://blahblah home-manager which won't work out of the box on non-NixOS systems.

Answer 16 · 2019-01-22T17:18:38.000Z

This broke my configuration and the commit message gives no indication about how to fix it. What is the proper way to stop this behaviour?

Answer 17 · 2019-01-22T19:05:36.000Z

@mpickering What commit are you referring to?

Answer 18 · 2019-01-22T19:10:18.000Z

8ab8d9cb74e75de46e647a5d77c137f7ef3ebd2b

I think I should fix it by removing all channels?

Answer 19 · 2019-08-30T01:44:24.000Z

Could someone explain the purpose of every user having:

NIX_PATH=nixpkgs=/nix/var/nix/profiles/per-user/root/channels/nixpkgs:/nix/var/nix/profiles/per-user/root/channels

as set up by the /nix/var/nix/profiles/default/etc/profile.d/nix-daemon.sh script in a multi-user non-NixOS install? Shouldn't the user just be adding/upgrading/using their own channels?

The problem right now is that if the user additionally sources ~/.nix-profile/etc/profile.d/nix.sh, there's a good chance the nixpkgs channel definition will be duplicated (so they'll always get that warning, which I only understand now after several hours looking through the various shell scripts and Github issues). If they source it after the /etc/profile.d stuff (which is likely), their duplicate channel definitions lose.

Would anything break if the root channels were only added to NIX_PATH for the root user?

Answer 20 · 2020-12-12T14:20:32.000Z

I just tried to install Nix on MacOS today. Steps to reproduce:

# Install multi-user on MacOS with auto-create of /nix volume
$ sh <(curl -L https://nixos.org/nix/install) --darwin-use-unencrypted-nix-store-volume --daemon

# After installation successful, I check /etc/zshrc and /etc/bashrc. Both have
# Nix
if [ -e '/nix/var/nix/profiles/default/etc/profile.d/nix-daemon.sh' ]; then
  . '/nix/var/nix/profiles/default/etc/profile.d/nix-daemon.sh'
fi
# End Nix

# Next I can check the $NIX_PATH of the new shells (both BASH and ZSH):
$ echo $NIX_PATH
nixpkgs=/nix/var/nix/profiles/per-user/root/channels/nixpkgs:/nix/var/nix/profiles/per-user/root/channels

It is all fine and game until I follow this step in the manual. Chapter 7 – Environment Variables.

I tried to add:

$ cat $ZDOTDIR/.zshrc.local -r 41:41
source '~/.nix-profile/etc/profile.d/nix.sh'

but everytime a new shell open, it showed this error: $HOME/.config/dotfiles/shell/zsh/.zshrc.local:.:41: no such file or directory: ~/.nix-profile/etc/profile.d/nix.sh.

I can run it after the shell has been initialized but in general, I don't know how to fix this.

Answer 21 · 2020-12-12T18:22:38.000Z

Hi @anhdle14 you'd run source ~/.nix-profile/etc/profile.d/nix.sh to initialize Nix if you were using a single-user installation.

The --daemon flag to the installer sets up a multi-user installation which puts that same initialization script at /nix/var/nix/profiles/default/etc/profile.d/nix-daemon.sh hence the lines you mentioned in etc/zshrc:

if [ -e '/nix/var/nix/profiles/default/etc/profile.d/nix-daemon.sh' ]; then
  . '/nix/var/nix/profiles/default/etc/profile.d/nix-daemon.sh'
fi

There shouldn't be any need to source '~/.nix-profile/etc/profile.d/nix.sh' which as the error points out doesn't actually exist. Is something not working with your setup? Are you trying to fix something by running that command?

Also when you say you can run it after the shell has been initialized what it "it"? Are you suggesting that the same command does not error with no such file... if you run it after shell startup?

Answer 22 · 2020-12-13T01:39:44.000Z

Hi, @nathanshelly! Thank for the quick response.

Also when you say you can run it after the shell has been initialized what it "it"? Are you suggesting that the same command does not error with no such file... if you run it after shell startup?

"it" here is the the script to change NIX_PATH during shell init lifecycle.

Yes, if I ran the script after the shell startup it can find the file. In any locations other than /etc/zshrc: .zprofile, .zshenv, .zshrc it won't run.

The reason I was fixating on using the nix.sh with nix-daemon.sh is because of $NIX_PATH.

I couldn't understand why installing something like nix-darwin always reported that they couldn't find darwin channel. It only work if I re-edit $NIX_PATH from

# From
export NIX_PATH=nixpkgs=/nix/var/nix/profiles/per-user/root/channels/nixpkgs:/nix/var/nix/profiles/per-user/root/channels
# To
export NIX_PATH=nixpkgs=/nix/var/nix/profiles/per-user/root/channels/nixpkgs:/nix/var/nix/profiles/per-user/root/channels:~/.nix-defexpr/channels

But the problems is I can't fix NIX_PATH within the shell init lifecycles. Now, every time I want to use darwin-rebuild I have to source it after the shell started.

Sorry I just learning Nix for 2 days, maybe I am doing something wrong but the documentations are pretty bad to follow.

Answer 23 · 2020-12-13T19:53:56.000Z

If I understand things correctly I believe your issues could be solved by adding the following line (which this issue is proposing should be included by default) to your $HOME/.zshrc (or other startup file depending on your preferences):

export NIX_PATH=$NIX_PATH:$HOME/.nix-defexpr/channels

I'm still learning Nix myself so I don't have context on why that is not the default already. Let me know if that doesn't work and we can try to keep debugging things 😄

Answer 24 · 2020-12-14T09:18:20.000Z

Thank you that did the trick, I thought i have to use nix.sh for correct tools per user, or else it only going to use the root one

Answer 25 · 2020-12-19T21:58:38.000Z

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/emacs-app-installed-incorrectly-via-home-manager/10577/2

Answer 26 · 2021-03-09T21:42:40.000Z

When I first faced this issue, I was disappointed and surprised but luckily I came up with a workaround that works for me:

prepare a script file

# /somewhere/path.sh

#!/usr/bin/env sh

cat << EOF > /etc/profile.d/user-shim-for-nix-path.sh
# the filename is intentionally behind `nix.sh`
export NIX_PATH=\${HOME}/.nix-defexpr/channels:\${NIX_PATH}
EOF

run sudo /somewhere/path.sh after nix installation
run source /etc/profile.d/user-shim-for-nix-path.sh when it's not sourced yet before any script using nix-*

I was able to automate my simple nix setup nicely with this way.

Answer 27 · 2021-08-21T17:46:01.000Z

Was there a resolution to this issue? I see that is was fixed in NixOS/nixpkgs@5d03cce but those changes were reverted in NixOS/nixpkgs@fa67ca1.

For reference, I just performed a fresh multi-user install on a new Mac (Big Sure 11.3.1) and $NIX_PATH is

$ echo $NIX_PATH
nixpkgs=/nix/var/nix/profiles/per-user/root/channels/nixpkgs:/nix/var/nix/profiles/per-user/root/channels

which does not include $HOME/.nix-defexpr/channels.

As best I can tell, $NIX_PATH is getting set by this line in /nix/var/nix/profiles/default/etc/profile.d/nix-daemon.sh (called in /etc/zshrc)

export NIX_PATH="nixpkgs=/nix/var/nix/profiles/per-user/root/channels/nixpkgs:/nix/var/nix/profiles/per-user/root/channels"

Answer 28 · 2022-03-29T06:34:05.000Z

Was unable to get $NIX_PATH with latest nix (2.7) and BigSur 11.6.4

Answer 29 · 2022-05-26T23:50:26.000Z

I had a friend install Nix today and NIX_PATH wasn't set on an aarch64-darwin install after using the default multi-user install (Nix 2.8).

Answer 30 · 2022-12-28T04:48:19.000Z

Installing a clean NixOS also currently (9a2e72335c6651d5fb16e6901aa4b5c790c243db) results in this broken state where channels added with nix-channel are not found by nix-env or nix-shell (and due to this, the home-manager install instructions also do not work).

What is the path to having a working user environment after a clean install of NixOS?

Answer 31 · 2022-12-28T23:09:32.000Z

It looks like most folks here are installing nix on non-NixOS, so this won't be relevant for most.

But for a NixOS install, I've ended up with the following workaround in the nixos configuration (with the added bonus of avoiding channels_root being added):

environment.shellInit = ''export  NIXPATH="/nix/var/nix/profiles/per-user/$USER/channels:nixos-config=/etc/nixos/configuration.nix"''