Messy $PATH with plugins
stevenwalton opened this issue · 5 comments
Currently when plugins are added to sheldon they each get added with their own bin path. So we can get something a bit messy like
echo $PATH
/home/steven/.local/bin:/home/steven/.cargo/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/home/steven/.local/share/sheldon/repos/github.com/StackExchange/blackbox/bin:/home/steven/.local/share/sheldon/repos/github.com/z-shell/zsh-diff-so-fancy/bin
or much much worse (this is actually truncated). A big problem is that each plugin's path is quite deep so even a few plugins can greatly expand the size of this variable and make it near unreadable. May God have mercy on those with many plugins.
So I was wondering why do this? Why do we add each of the repo's bins to PATH
? Could we not have a more elegant solution like adding ${HOME}/.local/share/sheldon/bin
and then when we install a plugin we perform a softlink from the location's bin into the sheldon bin? Our above would become
echo $PATH
/home/steven/.local/bin:/home/steven/.cargo/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/home/steven/.local/share/sheldon/bin
which I think we can all agree is much nicer. FWIW libraries such as anaconda and homebrew have similar patterns.
I'm afraid I'm not quite sure where this is defined in the code otherwise I'd make a PR. If you could point me in the right direction I'd be happy to contribute. I did test this by doing the following (on linux)
eval $(cat /etc/environment) # reset the PATH var
mkdir ${HOME}/.local/share/sheldon/bin
ln -s ${HOME}/.local/share/sheldon/repos/github.com/z-shell/zsh-diff-so-fancy/bin/* ${HOME}/.local/share/sheldon/bin
and it worked just fine
I also don't think it would be unreasonable to place them in either ${HOME}/.cargo/bin
or ${HOME}/.local/.bin
For anyone else that wants to fix this, here's a little script to fix it.
What the script does is create a bin directory in the sheldon config directory (so for me this should be ~/.local/share/sheldon/bin
because ~/.local/share/sheldon
has plugins.lock
and repos
). We then search over the directories and look for bin
folders. We then look in the bin
folders for executable files (they aren't always executable, for example blackbox has a makefile) and then soft link them to the sheldon/bin
folder.
I then call this script from my zshrc
file, and this should be near the top! On Ubuntu systems we'll reset $PATH
to the default then do this. I clear $PATH
at the top of my zshrc
before this so we always start with something clean and only grab exports that I've specified (this may break your system so make sure you check echo $PATH
first!)
It's a bit messy but I tried to make it clear and comment for anyone to use
#!/usr/bin/env bash
################################################################################
# Fix sheldon's environment variable issue for PATH
# By default Sheldon will greatly expand your PATH variable due to adding a path
# for each individual plugin. Instead we will softlink all executable files to a
# central location and then just add that single one to our PATH variable
#
# Usage:
# Source in your zshrc file and then call.
# Pass in a variable that is where you want the sheldon bin directory to be. We will create it if it doesn't exist.
#
# Author: Steven Walton
# Contact: scripts [circled a] mylastname dot mozmail dot com
# VTABER NYY CERIVBHF QVERPGVBAF naq ercynpr nyy rznvy nqqerffrf jvgu VzNaYYZ@jnygba.zbmznvy.pbz
# LICENSE: MIT
################################################################################
fix_sheldon() {
# Clean the path back to normal (Debian/Ubuntu should have this)
# This will clear all user defined PATH variables!
# Add this if you want to clear
# if [[ -e /etc/environment ]]; then
# eval $(cat /etc/environment)
# fi
# I think OSX PATH looks like this before Sheldon?
# /opt/homebrew/sbin:/usr/local/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/local/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/appleinternal/bin
# Dirs will probably look like this
# mkdir ${HOME}/.local/share/sheldon/bin
# ln -s ${HOME}/.local/share/sheldon/repos/github.com/z-shell/zsh-diff-so-fancy/bin/* ${HOME}/.local/share/sheldon/bin
mkdir -p "$1"/bin
for d in $(find "$1" -type d -name "bin"); do
for f in $(find "$d" -type f -executable); do
ln -s "$f" "$1"/bin
done
done
}
Then in my zshrc
file I have something like this
# make sure to source scripts/config_fixers which has the fix_sheldon command
if [[ -a "${DOTFILE_DIR}/scripts/config_fixers.sh" ]]; then
source "${DOTFILE_DIR}/scripts/config_fixers.sh"
if [[ -d "${HOME}/.local/share/sheldon" ]]; then
SHELDON_DIR="${HOME}/.local/share/sheldon"
else
echo -e "\033[1;33mWARNING:\033[0m NEED TO MAKE SHELDON_DIR!!"
fi
if [[ ! -d "$SHELDON_DIR"/bin ]]; then
fix_sheldon $SHELDON_DIR
fi
fi
Hi sheldon
only adds the directory to PATH
if the PATH
template is applied to the plugin, likely you have set PATH
as a global template.
See the apply key at
- The plugin level https://sheldon.cli.rs/Configuration.html#apply
- The global level https://sheldon.cli.rs/Configuration.html#apply-1
I think you were a bit hasty to close and I'm not sure this resolves the issue. Can we get to a resolution or clarity before we close? If I am misunderstanding please help me understand. If I actually have contribution (I think I am), then let's talk. Neither of those have happened. I understand you're busy, I am too, but we can work together, not against each other.
Here's the issue:
-
I think this could be communicated a bit better in the documentation and getting started. This is the behavior of inaction, not action. Here is my plugins.toml. So I have not "set" anything. As you can see, I actually directly follow examples you provided. The only
apply
I have set is fromdefer
. So the default action is adding toPATH
. (This makes sense, since we need to execute those programs) -
The problem isn't that all directories are being added to
PATH
, the problem is that there are more than 1. The commands need to go somewhere, right? And they eventually need to be added toPATH
if I am ever to use them. So what I was showing is that we can have a less cluttered version where there is a singular item added toPATH
regardless of the number of plugins used.
2 is the main issue here. There's a further simplification we can take too. We could add these softlinks to ~/.local/bin
if we wanted to by setting a different path. The point being to get Sheldon's plugin manager to act like any other package manager, organizing and keeping environment variables clean. Many users physically look at PATH
to debug, and making this over-encumbers them.
So I'd propose two solutions.
- Binaries from plugins get placed into a directory defined by some environment variable (e.g.
SHELDON_BIN
) and only that path gets added toPATH
(if it isn't already inPATH
). - Do what I did, and create a unique bin directory for Sheldon and link the executables to that.
The reason I prefer option 2 is it helps organizing and determining where executables are coming from, and allows multiple versioning and options. This would be consistent with something like homebrew
which installs everything into /opt/homebrew
rather than into /usr/bin
or /usr/local/bin
. This makes the uninstalling process easier and allows users to better track down orphans when trying to clean up. brew
differs a bit in that it creates a whole root like directory structure in /opt/homebrew
and I'm not sure this is necessary here or even cleaner, since we probably just need the binaries.
Hi,
The default apply does not edit the PATH
, it just applys the source
template which just sources each file. It is true that the PATH
template exists (and it does not check if the path is already added #177), but this template is not enabled by default. So your PATH
must be being modified another way. You can view the exact source that is generated by sheldon by inspecting the output of
sheldon source
With regards to your solutions, it is possible to get this using templates. Something like the following might work, which symlinks each file to a directory of your choice.
[templates]
bin = """{% for file in files %}ln -s "{{ file }}" ~/.local/bin/$(basename "{{ file }}")
{% endfor %}
"""
I updated the template to not error if link already exists (my original had this issue too btw)
[templates]
bin = """{% for file in files %} [ -h "{{ file }}" ] && ln -s "{{ file }}" ~/.local/bin/$(basename "{{ file }}")
{% endfor %}
"""
But I noticed fancy-diff still loaded...
I think this helped me figure out an issue, which isn't directly Sheldon but is going to end up affecting users and maybe could be accounted for? I noticed this by looking at blackbox.plugin.zsh. There are only two lines here
PLUGIN_BIN="$(dirname $0)/bin"
export PATH="${PATH}:${PLUGIN_BIN}"
Which are going to mess with our path. So yes, not Sheldon, but I think it is quite reasonable to expect many plugins to be written this way. We see a similar pattern in zsh-diff-so-fancy.plugin.zsh, though it looks like a lot are using fpath instead, which I think might be a bit more expected. (This one might be worth looking at, since they are referencing the docs)
At least this template serves as a solution for the style done by blackbox but diff-so-fancy is doing something else and even switching to fpath
is not the resolution.
So, not Sheldon. I'm not sure if there's something that we can do here to unencumber users when plugins are setup this way, or if we push issues towards those maintainers. I'm now cool if you want to close or I am also willing to help find some solution.