Misterio77/flavours

Cannot run `update all`

Sleepful opened this issue · 19 comments

Sleepful commented

cryptic error, .config/flavours does not exist after running the flavours update all, I do see stuff inside ~/Library/Application\ Support/flavours/base16

/usr/share/doc/git/contrib/credential/libsecret/git-credential-libsecret get: /usr/share/doc/git/contrib/credential/libsecret/git-credential-libsecret: No such file or directory
fatal: could not read Username for 'https://github.com': terminal prompts disabled
fatal: cannot change to '/Users/user/Library/Application Support/flavours/base16/schemes/solarized': No such file or directory
/usr/share/doc/git/contrib/credential/libsecret/git-credential-libsecret get: /usr/share/doc/git/contrib/credential/libsecret/git-credential-libsecret: No such file or directory
fatal: could not read Username for 'https://github.com': terminal prompts disabled
fatal: cannot change to '/Users/user/Library/Application Support/flavours/base16/schemes/ia': No such file or directory
Error: Git failed to run on repository 'https://github.com/aramisgithub/base16-ia-scheme'. Check if your repo list is valid.
Sleepful commented

btw the latest release does not include the latest commit, also you do not offer instructions to install from source so I do not know how to go about that

Sleepful commented

I guess this works for now :S

cd "~/Library/Application Support/flavours/base16"
gsed -i '/aram/d' sources/schemes/list.yaml
flavours update templates
flavours update schemes
  • 👍2
Sleepful commented

also I have to pass in the config path or it wont find it:

flavours apply nord

Error: Couldn't get items from config file. Check the default file or github for config examples.


flavours --config ~/.config/flavours/config.toml apply nord
:ok
Sleepful commented

Re last comment:

I had to pass in --config because XDG_CONFIG_HOME was not available, a better error message would be helpful :)

vladnick31 commented

Have the same problem and ^ these fixes doesn't seem to help.
flavours update schemes is what is giving the error. And I feel stuck not being able to use the program in any way. Really want to, though.

titaniumtraveler commented

Have the same problem and ^ these fixes doesn't seem to help. flavours update schemes is what is giving the error. And I feel stuck not being able to use the program in any way. Really want to, though.

It seems that a few of the scheme sources are no longer available, breaking the update process.
Specifically ia: https://github.com/aramisgithub/base16-ia-scheme and solarized: https://github.com/aramisgithub/base16-solarized-scheme.
(When trying to access the github I only get a 404.)

Fixing this should be as easy as commenting both out in $XDG_DATA_HOME/flavours/base16/sources/schemes/list.yml.
(Where $XDG_DATA_HOME defaults to $HOME/.local/share, so $HOME/.local/share/flavours/base16/sources/schemes/list.yml.)

Though if you use flavours update list, (or flavours update all, which internally calls this,) you will have to modify it again.

  • 👍2
vladnick31 commented

Yeah I guess this worked for the schemes to update. Still when trying to apply any scheme it tells me there are no .mustache files in the .config/flavours/templates"..." folder. Even though every folder has a "template.yaml" file. Must have configured something wrong I guess.

MrPenguin07 commented

Just got hit by this, also building from source.

Project seems unmaintained how unfortunate. May have to maintain this locally.

titaniumtraveler commented

Just got hit by this, also building from source.

Project seems unmaintained how unfortunate. May have to maintain this locally.

I mean flavours uses https://github.com/chriskempson/base16-schemes-source as default source for schemes, which is kind of badly maintained.

See:

flavours/src/operations/update.rs

Line 48 in 884dadc

let default_s_repo = "https://github.com/chriskempson/base16-schemes-source.git";

MrPenguin07 commented

Just got hit by this, also building from source.
Project seems unmaintained how unfortunate. May have to maintain this locally.

I mean flavours uses https://github.com/chriskempson/base16-schemes-source as default source for schemes, which is kind of badly maintained.

See:

flavours/src/operations/update.rs

Line 48 in 884dadc

let default_s_repo = "https://github.com/chriskempson/base16-schemes-source.git";

Indeed, ideally this external list should be forked and under flavours control (considering a simple 404 on this list breaks initial setup here!) or simply integrated into flavours however I see the author was made aware of this quite some time ago and seems uninterested?
Would a PR be merged?

titaniumtraveler commented

Indeed, ideally this external list should be forked and under flavours control (considering a simple 404 on this list breaks initial setup here!)

You can actually do that yourself! While it is not really documented, you can override the default by either

  • setting schemes/templates in the config.toml
  • setting schemes/templates in $XDG_DATA_HOME/flavours/base16/sources.yaml (defaults to $HOME/.local/share/flavours/base16/sources.yaml)

See the get_sources function:

flavours/src/operations/update.rs

Lines 46 to 98 in 884dadc

fn get_sources(file: &Path, config: &Config) -> Result<(String, String)> {
// Default repos
let default_s_repo = "https://github.com/chriskempson/base16-schemes-source.git";
let default_t_repo = "https://github.com/chriskempson/base16-templates-source.git";
// config source indicators
let use_config_sources = config.schemes.is_some();
let use_config_templates = config.templates.is_some();
// use available config sources, otherwise use defaults
let (mut s_repo, mut t_repo) =
match (config.schemes.as_ref(), config.templates.as_ref()) {
(Some(s), Some(t)) => (String::from(s), String::from(t)),
(Some(s), None) => (String::from(s), String::from(default_t_repo)),
(None, Some(t)) => (String::from(default_s_repo), String::from(t)),
(None, None) => (String::from(default_s_repo), String::from(default_t_repo)),
};
// Try to open file
let sources_file = match File::open(file) {
// Success
Ok(contents) => contents,
// Handle error once, so if file is not found it can be created
Err(_) => {
// Try to write config sources if they exist, otherwise use defaults
write_sources(&s_repo, &t_repo, file)?;
// Try to open it again, returns errors if unsucessful again
File::open(file).with_context(|| format!("Couldn't access {:?}", file))?
}
};
// Bufreader from file
let reader = BufReader::new(sources_file);
// Iterate lines
for line in reader.lines() {
let line = line?;
// Get name and repo from line
let (name, repo) = parse_yml_line(&line)?;
// Store in correct variable
// Only use sources.yaml sources if sources not specified in config
if name == "schemes" && !use_config_sources {
s_repo = repo;
} else if name == "templates" && !use_config_templates {
t_repo = repo;
}
}
// Rewrite file using found repository, this is done to clean up errors on the file or insert default values
write_sources(&s_repo, &t_repo, file)?;
// Return repos
Ok((s_repo, t_repo))
}

Otherwise it might be a good idea to have something like --ignore-errors, which ignores errors like the 404 and just does what it can.

or simply integrated into flavours however I see the author was made aware of this quite some time ago and seems uninterested?

In my opinion that would be kind of out of scope for flavours.
Having functionality to configure where the sources come from. Sure.
Managing the sources itself. Not really.

Would a PR be merged?

I can't speak for the maintainer, but I would guess that while it might take a while, that yes.

MrPenguin07 commented

Indeed, ideally this external list should be forked and under flavours control (considering a simple 404 on this list breaks initial setup here!)

You can actually do that yourself! While it is not really documented, you can override the default by either

* setting `schemes`/`templates` in the `config.toml`

* setting `schemes`/`templates` in `$XDG_DATA_HOME/flavours/base16/sources.yaml` (defaults to `$HOME/.local/share/flavours/base16/sources.yaml`)

See the get_sources function:

flavours/src/operations/update.rs

Lines 46 to 98 in 884dadc

fn get_sources(file: &Path, config: &Config) -> Result<(String, String)> {
// Default repos
let default_s_repo = "https://github.com/chriskempson/base16-schemes-source.git";
let default_t_repo = "https://github.com/chriskempson/base16-templates-source.git";
// config source indicators
let use_config_sources = config.schemes.is_some();
let use_config_templates = config.templates.is_some();
// use available config sources, otherwise use defaults
let (mut s_repo, mut t_repo) =
match (config.schemes.as_ref(), config.templates.as_ref()) {
(Some(s), Some(t)) => (String::from(s), String::from(t)),
(Some(s), None) => (String::from(s), String::from(default_t_repo)),
(None, Some(t)) => (String::from(default_s_repo), String::from(t)),
(None, None) => (String::from(default_s_repo), String::from(default_t_repo)),
};
// Try to open file
let sources_file = match File::open(file) {
// Success
Ok(contents) => contents,
// Handle error once, so if file is not found it can be created
Err(_) => {
// Try to write config sources if they exist, otherwise use defaults
write_sources(&s_repo, &t_repo, file)?;
// Try to open it again, returns errors if unsucessful again
File::open(file).with_context(|| format!("Couldn't access {:?}", file))?
}
};
// Bufreader from file
let reader = BufReader::new(sources_file);
// Iterate lines
for line in reader.lines() {
let line = line?;
// Get name and repo from line
let (name, repo) = parse_yml_line(&line)?;
// Store in correct variable
// Only use sources.yaml sources if sources not specified in config
if name == "schemes" && !use_config_sources {
s_repo = repo;
} else if name == "templates" && !use_config_templates {
t_repo = repo;
}
}
// Rewrite file using found repository, this is done to clean up errors on the file or insert default values
write_sources(&s_repo, &t_repo, file)?;
// Return repos
Ok((s_repo, t_repo))
}

Otherwise it might be a good idea to have something like --ignore-errors, which ignores errors like the 404 and just does what it can.

or simply integrated into flavours however I see the author was made aware of this quite some time ago and seems uninterested?

In my opinion that would be kind of out of scope for flavours. Having functionality to configure where the sources come from. Sure. Managing the sources itself. Not really.

Would a PR be merged?

I can't speak for the maintainer, but I would guess that while it might take a while, that yes.

Ofcourse it can be done by oneself. That's not the issue.

The issue is that new users are being hit with this upon install, and spending who knows how long figuring it out.
Then hacking their way around it to get it working.

The correct solution is as I stated previously.
It's a very quick 'fix' - for an otherwise interesting project seems odd the author hasn't done so or nominated someone else.

Nb. not intended to be an attack on the author. We are free to fork and whatnot and thanks for your work thus far.

Misterio77 commented

Hey all, sorry for the radio silence. Life's been busy and flavours is not something I use every day, so this ended up being low priority.

This problem with sources is something that actually drives me a bit away from base16, and, IMO, demands a flavours refactor.

In my opinion, the source listings are bad design. Users are super likely to want to include custom schemes or templates, so most people who use flavours for over a day probably have their own source listings fork, don't update lists, or use the under-documented extra_{scheme,template} feature.

Tinted theming's approach to having all schemes under a single repo is better, but switching flavours to it without breaking people's workflows would be absolutely impossible (or very darn hard).

If we want to do breaking changes, I'd like to make it just once and have the best design possible, so that we avoid breaking in the future again.

Flavours biggest design problem ATM is the way it manages sources. Source listings and keeping stuff on ~/.local/share is a footgun. We have gradually added features for better managing these (which most power uses use, as I mentioned). We should strip the footgun features altogether, and elevate the prefered solutions to be the preferred usage.

My ideal solution looks like this:

  • Stuff on ~/.local/share/flavours should not be supposed to be edited, at all, they are harder to git-track when compared to a single config. Let's make it so that any and all sources are configured through the config file.
    • I'd argue that downloaded schemes and templates should actually go to ~/.cache, and stop storing persistent data althogether.
  • Schemes are not usually modified (only added). For the best UX, schemes should, by default, come from a single (configurable, defaulting to tinted theming's) repository.
    • Extra schemes are to be added through extra_schemes (like we have today).
    • For better usability, also keep support for reading schemes from ~/.config/flavours/schemes
  • Templates are usually more coupled to a users' config (not all programs have variables on their configs, for example), so people frequently (even more than schemes) fork them. This means that having a source listing for them is not worth it. Let's make it so that every template entry must specify its source (either remote or local file).

This is how a flavours 2.0 config file would look like with that design:

shell = "bash -c '{}'"

[schemes]
source = "git+https://github.com/tinted-theming/base16-schemes" # If set to null, flavours will ignore it
extra = ["https://github.com/foo/bar/tree/blob/foo.yml"] # List of remote yml files

[[items]]
file = "~/.config/sway/config"
source = "https://github.com/rkubosz/base16-sway/blob/master/templates/colors.mustache"
hook = "swaymsg reload"
light = false

[[items]]
file = "~/.config/waybar/colors.css"
source = "https://github.com/mnussbaum/base16-waybar/blob/master/templates/default.mustache"
rewrite = true

[[items]]
file = "~/.config/beautifuldiscord/style.css"
source = "https://github.com/samme/base16-styles/blob/master/templates/css-variables.mustache"
start= "/* Start flavours */"
end = "/* End flavours */"

[[items]]
file = "~/.config/foo/config"
source = "~/.config/flavours/templates/foo.mustache" # Local files are also supported

In my opinion, this is a huge breaking change and should warrant flavours 2.0. I think maintaining support for the old and new ways is pretty much impossible (there's a LOT of edge cases, so it's impossible to not break anything). We should instead try and detect legacy users and point them to a (easy and quick to follow) migration guide. I thought of this:

  • Remove update subcommand
    • If users try to run it, error out and point to the migrating guide.
  • Flavours will download schemes and templates if needed while applying, if they don't exist in ~/.cache/flavours.
    • Add a flavours cache-clear to make it easy to clear the cache.
    • Add a --offline switch, for usecases where flavours should never try to connect to the internet.
      • This may not be needed, as a user with a config without any remote template sources nor remote extra schemes, and with scheme source set to null will not download anything.
    • A drawback is that flavours list will not work unless there is cached schemes.
  • Entries without explicit template sources are to be deprecated. Error out and point users to migration guide.
  • extra_templates are to be deprecated, as they are not needed anymore.
  • Schemes will now come from tinted theming repo by default.
    • Warn the user if the same scheme exists in the legacy locations?
    • "Scheme not found" errors should also contain a link to the migration guide.
  • Maybe a flavours migrate command that tries and migrate them automagically?

Well, that took a while to type. This is overall my vision for a improved flavours, and I would like to hear your thoughts on it. I will look into opening a tracking issue, a branch, and draft PR for it.

Of course, this is a huge undertaking and requires a lot of refactoring. I'm currently working full time and doing a master's; it's hard to have time to do open source work at all, and when I do, it's usually programs I use daily. I do realise flavours has a huge community and deserves more love for me, so I'll do my best.

PRs are totally accepted, but I'll refrain from merging new features to the legacy version for now.

Would a PR be merged?

I can't speak for the maintainer, but I would guess that while it might take a while, that yes.

Ofcourse it can be done by oneself. That's not the issue.

The issue is that new users are being hit with this upon install, and spending who knows how long figuring it out. Then hacking their way around it to get it working.

As you can see, the solution is not a one-liner fix, but a deeper design problem. People have different usecases than I anticipated when I built this, and base16 has evolved into a painful to deal with standard (with lots of flaws).

The correct solution is as I stated previously. It's a very quick 'fix' - for an otherwise interesting project seems odd the author hasn't done so or nominated someone else.

I don't want to sound rude, @MrPenguin07, but if that is "quick fix", do feel free to open a PR. I am sure you will run into the design flaws when you try and implement it. People (myself included) have def tried to tackle this problem, and they always give up due to running into the flaws I mentioned.

All fixes are easy when you don't consider existing architecture, users that rely on that behaviour, etc.

Back on topic. Fixes while flavours is not re-written to tackle the design flaws:

  • As far as I have tested it, the download failures should not be fatal, flavours should keep going. For anyone that flavours did stop after a failure, can you reply to me so that we can try and reproduce it?
  • I will look into forking the sources, but I can't guarantee I will be able to make them as up to date as they should be.
MrPenguin07 commented

I don't want to sound rude, @MrPenguin07, but if that is "quick fix", do feel free to open a PR. I am sure you will run into the design flaws when you try and implement it. People (myself included) have def tried to tackle this problem, and they always give up due to running into the flaws I mentioned.

All fixes are easy when you don't consider existing architecture, users that rely on that behaviour, etc.

Back on topic. Fixes while flavours is not re-written to tackle the design flaws:

* As far as I have tested it, the download failures should not be fatal, flavours should keep going. For anyone that flavours did stop after a failure, can you reply to me so that we can try and reproduce it?

* I will look into forking the sources, but I can't guarantee I will be able to make them as up to date as they should be.

Appreciate the response, and the thought you've put into restructuring for a 2.0 release - it's a solid plan.
I also don't mean to seem rude - it can be difficult to provide constructive criticism from an end users perspective after running into issues during install.

I am sure you will run into the design flaws when you try and implement it. People (myself included) have def tried to tackle this problem, and they always give up due to running into the flaws I mentioned.

I have to disagree, can't help but think you're overthinking the obvious solution(s) which would provide immediate relief for new users just finding flavours. Your high level view of these flaws is spot on, however this is from the perspective of a new user trying to get up and running.
The crux of this particular issue is very simple - new users follow the INSTALL, and while running update script to populate the templates are left with a somewhat cryptic error, no templates are installed hence users give up in frustration.

All due to 404's caused by another github account which has been deleted and the update script has no error handling and dies. At minimum these accounts are no longer accessible.
https://github.com/aramisgithub, https://github.com/Whillikers

Solution 1) add error handling for the 404's to simply skip those repos and allow update to populate.
Solution 2) fork or otherwise absorb default_s/t_repos, remove the offending templates and update the scripts link.

Neither solution should be a breaking change and affect existing users. Unless i'm overlooking something?
I'll consider a P.R to do more than simply complain, only issue is I am one of the (I suspect many) who moved on after this issue.

titaniumtraveler commented

@Misterio77
I have a lot of thoughts on this how this refactor could be achieved,
though currently they are pretty unordered and hard to express in words.
(I have one big unordered markdown document with a lot of random ideas,
but currently in a format where I doubt anyone else could read/understand it.)

The main points are the following:

  • We can keep backwards compatibility without having too much code complexity by essentially having two configs:
    • An "outer" config, which takes in the user config and is fairly liberal in terms of what it accepts.
    • An "inner" config, which is created based on the old one and has no weird edge cases and is easier to work with.
    • A simple function that does the translation between the two and maybe does some deprecation warnings if necessary.
  • Have a lock-file:
    Being able to recreate the state from scratch is really useful
    and I think it gives way to maybe even implement the ability to set different items to different color-schemes.
  • Using libgit2 for fetching the data.
    That way we can easier do things like monitoring the fetch-progress,
    using a specific version of a scheme/template (by setting something like a ref-field)
  • Supporting both the source listings and tinted-theming variant:
    The repo format could be either recognized automatically (by testing if list.yaml exists for example) or explicitly configured to be in one specific format.

I think I will start by implementing a prototype in its own repo, which then can be integrated into main flavours later.

Sleepful commented

The fundamental band-aid over this problem should be akin to "try catch" and not error on the 404 links. This should be an easy non-breaking change to the main repo: At least it makes it very simple for new users to adopt the tool right now, even if it isn't perfect.

Sleepful commented

For a simple solution that doesn't degrade as themes go 404, I would just vendor the current themes, i.e. download all the source files and use that as truth, maybe in a separate repo. At least that gives a good starting gallery to new users and the gallery doesn't break.

  • 👍1
Sleepful commented

Anyway, just for the record. After overcoming the hurdle of installation, flavours has been running like a charm in my terminal, it has allowed me to enjoy coding while keeping my eyes sane for the past year or so! Awesome work.

I made a tmux CMD to pick themes with fzf-tmux, in case anyone wants to try it:

https://github.com/Sleepful/.config/blob/master/.tmux.conf#L87