Decide how to handle universal template builders
belak opened this issue · 40 comments
In order to simplify the average builder (and allow them to focus on the building portion rather than git operations), we replaced the base16-schemes-source repo and all the sub-repos with https://github.com/base16-project/base16-schemes.
We also dropped the base16-templates-source repo for the same reason - most builders are focused on building a single template at a time.
Unfortunately, when I wrote spec 0.10.0, I didn't consider theme managers like themix and it wasn't brought up.
I'm not sure what the best course of action is - I don't like the old setup where multiple repos have to be updated for a template to be submitted (the old setup also doesn't allow for a repo to be replaced if the maintainer becomes inactive). Eventually, I hope most of the repos are transferred to base16-project and base16-project-contrib, but until then we can figure something out.
Does anyone have any suggestions?
CC @actionless
from my side is really no difference to download themes/template from one repo or multiple:
you saw the code itself of that tool - is rly primitive 20-ish lines of code and easy to change to adapt to anything
but i'm not sure if all the scheme/template developers would be excited making a PR to some repo for every minor change:
so i think having some schemes/themplates as part of base16 project repo and some as third-party repos is totally fine so i'd suggest to move https://github.com/chriskempson/base16-schemes-source and template source to base16-project as well
for builder parsing YML and doing git-clone are inevitable steps anyway - so i'm not sure it's really a big simplification
@Misterio77 may have some good insight into this topic since they've created flavours, a manager/builder for base16.
Hey folks!
Thanks for the ping! When maintaining flavours and trying to understand what workflows are most popular, I've come to a few (maybe strong) thoughts.
EDIT: I got a bit excited and wrote a lot of stuff, perhaps some a little offtopic, should I repost this to another issue?
TLDR at bottom.
About the repos:
I absolutely loved the move to a single scheme repo.
Schemes are clearly something where the more the merrier, they don't usually conflict, most people want easy access to ALL of them, and having quality control is great. So it makes perfect sense for a single repository be the official source of truth.
About templates, I think they should indeed always just have their own repo, as belak mentioned.
They are very personal and users who use managers will always want to tweak at least a few of them. So, keep each in their own repo, no index repository (I think it's great we dropped it).
A few blessed ones on the main org, some on the contrib org, a lot on their creators github account. We can handle discoverability by offering to move them to our org, otherwise no "official unofficial" list or stuff like that.
This makes changing (for example, following a fork) template sources very easy.
I understand that the spec still keeps the entry about builders managing stuff, but I don't really agree. Builders should build, managers should manage.
I think builders should be simple software, designed for scripting (aka plumbing). They should primarily expose their functionality as CLI (with the interface catered for easy CI usage, and exactly the same for ALL builders), and optionally expose their functionalities as a software library (this helps building managers, our porcelain)
Exact same CLI interface, exact same features. They should be 100% drop in compatible between each other. The only difference being language, which users shouldn't care about (they won't even use them), just developers who plan on consuming the lib with their favorite language (for building managers, for example). We should have the lib for all languages possible!
With this out of the way, I'd argue builders should just have a single use: build. No git, no user data dir management, just build
Maybe something like this:
base16 template/config.yaml scheme1.yaml scheme2.yamlBuilds the templates specified on the given file with every given scheme. That's it. No pre configured scheme source, nor vendoring or cloning, or any implicit or networked behavior.
Example CI usage for a template repo:
git clone https://github.com/base16-project/base16-schemes
base16 --spec=1.0.0 template/config.yaml base16-schemes/*See what I mean? It being plumbing makes it easy to use with their intended use: as a component implementing the standard.
Base16 has two main use cases we should consider:
-
Users who just want a built template should go to the repo and grab it. I think these mostly don't care (and should not care) about builders and managers. Just simple grabbing and using, easily tweakable.
-
Users who frequently change themes should use a manager (which probably consumes a builder lib I mentioned), or another software who otherwise help using the base16 standard and ecosystem. These have opinionated workflows and can (and should) vary wildly! A few examples: CLI manager, GUI manager, website theming script, IDE theming feature, Nix module, etc.
Hope this was clear and readable, and hope someone agrees with it!
What does this means to managers
Builders, by the way, should absolutely not care about this. Managers, on the other hand, can do whatever they want! They should dictate how users (outside of the "go to repo and download theme you like" usecase) manage their templates.
For example, flavours' configs could expect repository URLs (such as github:example/template, or relative file paths) instead of name. For example:
schemes_source = "base16-project/scheme-source"
[sway]
template = "base16-project/sway-template"
file = "~/.config/sway/config"
subtemplate = "colors"
hook = "swaymsg reload"
[waybar]
template = "foo-bar/waybar-template"
file = "~/.config/waybar/colors.css"
rewrite = trueSee? Switching to forks (something flavours users always want to do, but modifying source yamls and overriding stuff is awful) is absolutely trivial. Official and contrib themes are highly discoverable, and unofficial ones are a github search away.
Keeping up to date with new schemes would also be really simple! This is very awkward with flavours ATM, specially when you have custom stuff, flavours update is scary so people end up putting their entire scheme and template dirs on git! This is awful, why version control so much stuff if just a flavours configuration file would be sufficient?
And, you noticed, this is fully compatible with existing template repositories!
Phew, long write.
TL;DR: Builders should just build, and be optimized for scripting (with a stable and uniform interface) and lib usage. Managers should dictate the workflow. Central schemes are great. Templates should just be repositories on our orgs or not, the manager will help them download and use their desired templates
Results: Easy and predictable CI usage. Easy manager implementation. Trivial builder implementation. Uniform usage of base16 libs across languages (or scripting), so people can build awesome stuff without thinking about how they will implement `{{ base00-dec-g }}|!
Thanks for reading, hope this crazy writeup makes sense and help somehow with future ideas.
I plan on writing an example Rust builders, and perhaps making improvements to flavours based on these. Keep an eye open if you're interested.
The node builder just looks into:
$CWD/base16/schemes/*.yaml$CWD/base16/templates/*/template
It doesn't care how the schemes or templates get there... For single template maintainers it's pretty easy, just git clone two repos (schemes & your template)... If someone wants to build 100 templates (at once) they just have to checkout all those repos (there is no automated facility to do that).
If someone had to script a builder in a loop though per template that wouldn't bother me, but I already had the multi-template build functionality so I just left it after ripping out all the git checkout stuff...
they just have to checkout all those repos (there is no automated facility to do that).
it's not a problem how to check out 100 repos :)
the problem here what before it was an explicit list of available templates: https://raw.githubusercontent.com/chriskempson/base16-templates-source/master/list.yaml while there is no anymore machine-parseable source of truth for this
Is a master list necessary? I think it's a very small minority of people who want/need to build "ALL the templates"... can we push maintaining that "master list" onto those few groups? I do think it'd be nice to have a master list in a README/wiki or somewhere for people to "discover" new programs that work with Base16... but that wouldn't have to be machine readable (though it could also be, killing two birds with one stone).
Even a Markdown list of Markdown links (pulled right out of a README) shouldn't be too hard to parse for most languages...
@joshgoebel how maintaining such list in a readme is easier than maintaining literally same list but in yaml as it was for years in original project before fork?
i think in a readme you could just put a link to that file, for people who want to manually discover the list of the templates instead of using any of tools who were supporting this for original base16
how maintaining such list in a readme is easier
A README (or other markdown doc) is easier to read/scan and more accessible to humans, which is the audience I think we should most care about.
Example (in table form):
A README (or other markdown doc) is easier to read/scan and more accessible to humans, which is the audience I think we should most care about.
YML format is used for both colorschemes and template, which makes that point not stand any critiques
but could you please elaborate what's so "inaccesible" in reading this list for humans (i even disabled the syntax highlight to make it worse):
P.S. considering what main target audience for base16 tools are coders and *nix-theming enthusiasts, tbh i think assuming they wound be not in ease reading yaml of such structure is sorta offending their intellect
can't read yaml
Please don't put words in my mouth. I said "easier to read"...
If you don't see how the table example I linked to is more immediately readable and scan-able (at a glance) then nothing I say is likely to persuade you, so I won't try. I'll let readers of this thread come to their own conclusions.
My point was that a centralized table in a README works well in other projects to solve a similar problem (discovery of 3rd party repos) - and hence it might also work well for us.
even if assuming what markdown would be easier to read than yaml list:
what would be easier for such easy user which you trying to paint here - to a) manually copy-paste the things to some config or even manually clone some repo or b) just choosing the theme in dropdown in gui tool or running builder-cli --list-templates, etc?
Please remember to keep it civil - it's clear there are some strong opinions here. Try to take the approach that everyone here has the best interests of the project in mind, even if they have different goals.
Please don't put words in my mouth
i've edit my reply, to prevent you from such impression
but could you please elaborate what's so "inaccesible" in reading this list for humans
There are plenty of things about that yaml that's not as accessible to humans, especially for non-programmers:
- Each "column" of data isn't aligned
- The link to the repository isn't clickable
- Additional data would have to be added as "keys" which gets even less readable and isn't as clear as columns in a table
- It requires users to understand the concept of "comments" in code, which isn't necessarily the case (as an example, I know a number of people who use Emacs are writers, not software developers)
One possible option would be providing a master list in yaml form and using it to generate a portion of the readme.
In general I tend to agree (though I'm biased since I'm the one who removed the base16-*-sources repos from the spec). At least for now, our goal is to support single template maintainers with the builder and let users install those templates themselves.
@actionless can you give a sample use-case for wanting to check out all the template repos and building them? The only use-cases I can think of involve automatically installing templates which would generally require the builder/manager to understand each template and where it goes and would need to be done on a template by template basis anyway (so a programmatic list wouldn't really help).
@actionless I think we're talking about two very different problems:
- Ease of a machine building/fetching/listing all templates
- A human reader learning/discovering what programs support Base16 themes
I'm proposing a solution that works well for 2 yet is also easily usable (with a simple URL fetch/regex) for 1... where-as I don't think a key: url YAML file solves need 2 quite as nicely at all. Although one could perhaps build a NICE Markdown file/table from the yaml file (if more meta-data was added)... tbut right now I find it quite lacking the meta-data I'd like to see in a central list (no author names, no human readable theme names, only slugs, etc)...
Though again I think case 1 is a bit of an edge case...
- I think consumers of base16 far outweigh maintainers... (easily provable I think)
- and consumers and maintainers (of a single theme) far, far outweigh services choosing to "build the base16 world" or maintainers of multiple themes.
I personally see "build the world" as a very specialized edge case.
There are plenty of things about that yaml that's not as accessible to humans, especially for non-programmers:
@belak Thanks for jumping in here with a list. :-)
One possible option would be providing a master list in yaml form and using it to generate a portion of the readme.
Ha, great minds! 🧠
One possible option would be providing a master list in yaml form and using it to generate a portion of the readme.
yes, i also think this would be the best option
@actionless can you give a sample use-case for wanting to check out all the template repos and building them? The only use-cases I can think of involve automatically installing templates which would generally require the builder/manager to understand each template and where it goes and would need to be done on a template by template basis anyway (so a programmatic list wouldn't really help).
just choosing the theme in dropdown in gui tool or running smth like builder-cli --list-templates, it's not only easier but also doesnt need internet (as all themes and templates are downloaded before the packaging):
also for such usecase it would be useful to have smth like description field (and mb some other metadata) there:
https://github.com/aarowill/base16-alacritty/blob/master/templates/config.yaml
UPD: and suggested output directory path (unless existing field output was already meant for that and being misused - i can't really figure that part out yet)
just choosing the theme in dropdown
Templates aren't themes... I can see where in say VS Code you might want to list all Base16 themes/schemes (to pick one)... but I'm not sure why you'd need a picker of all templates?
but I'm not sure why you'd need a picker of all templates?
to not pick them manually from some web page and download them one by one with hands, but to have them already there and just to pick from dropdown which user needs for their setup - if they need theme for vscode or for terminal or for both
I'm starting to maybe understand your use case:
Oh this Base16 theme in VS Code is just so cool - I think I want to install it in Terminal and Vim as well.
I'm not sure that problem is best solved by downloading templates one by one (regardless of how easy) and building themes by hand... and as @belak pointed out it's not super useful to just build the themes yet not install them, and installing could be a whole can of worms...
I'd wager (though I can't back this up) that many/most theme consumers are going to download the themes from "cooked" repos... not download template/builder and build them by hand.
pointed out it's not super useful to just build the themes yet not install them
installation is not yet supported and can't be supported before this will be addressed in template spec:
and suggested output directory path (unless existing field output was already meant for that and being misused - i can't really figure that part out yet)
in other words - you can't install it nowhere unless you have the medata for generating (or rather suggesting/defaulting) the output name
I'd wager (though I can't back this up) that many/most theme consumers are going to download the themes from "cooked" repos... not download template/builder and build them by hand.
for this usecase (creating cooked-up repo with many app-configs for your color-scheme) having universal template builder with all the templates at once actually would be useful, however i was never considering such usecase in mind
Sorry for cross post, but this belongs here:
I'm becoming less and less enamored with a builder spec... esp at the level actionless is desiring (and what perhaps we had before)... I think our goal (over time) should be to preserve compatibility with converting schemes into themes using templates generally speaking... not that the exact command line arguments and process for doing so should forever be set in stone.
I find this an onerous requirement and I honestly don't feel like the project or it's contributors have that kind of time.
I think if someone wants to maintain such a "universal builder" they should do it with some small wrapper scripts around one of the official builders... I don't think it needs to be baked into the Base16 spec itself.
If enough people found that useful those scripts could even live here... I just don't think it needs to be part of the spec.
installation is not yet supported
If you asked me personally this is out of scope for the project and if anything more a discussion first for template authors (that might circle back to builders later)...
(creating cooked-up repo with many app-configs for your color-scheme)
That's what several/many of the existing templates do - publish the cooked themes in the same repository as the template... for example: https://github.com/highlightjs/base16-highlightjs
That's what several/many of the existing templates do - publish the cooked themes in the same repository as the template... for example: https://github.com/highlightjs/base16-highlightjs
the counter-example of the opposite approach: https://github.com/joshwlewis/base16-unikitty (sorry i first put alacritty instead of unikitty, as both sounded similar to me)
the rest is only your subjective opinion not really backed with anything which would explain why tidy machine-readable list of templates must be downgraded to chaotic "human-readable" (i've already pointed out why i am ironically puting this into quotes) text document
i think previously-proposed idea for generating any user-facing docs from yaml should fit all possible usecases, whilst implementation of such GH-Action is more than trivial
the rest is only your subjective opinion not really backed with anything which would explain why tidy machine-readable list of templates must be downgraded to chaotic "human-readable" (i've already pointed out why i am ironically puting this into quotes) text document
This is a bit passive aggressive. We have already given a number of reasons of why we've focused on the human readable list before adding back the machine-readable one:
- We primarily cater to people who use the base16 themes: there are many more people using these themes than maintaining them.
- There hasn't been a strong use case made for software which generates all themes. The primary reason to generate multiple themes would be something which installs them as well and they'd need a custom method of supporting different themes anyway (installing the emacs theme is very different from installing the vim theme, etc).
the counter-example of the opposite approach: https://github.com/joshwlewis/base16-unikitty (sorry i first put alacritty instead of unikitty, as both sounded similar to me)
This is a color scheme rather than a template. I'm aware there are templates which don't provide generated files, but as we migrate more schemes into this org, we've been setting up CI which automatically builds and commits all the generated themes. This is most likely the path we'll continue on for now, so that is our primary use case, hence why we focused on that first.
i think previously-proposed idea for generating any user-facing docs from yaml should fit all possible usecases, whilst implementation of such GH-Action is more than trivial
This does seem fairly reasonable. I think the discussion has been focused around why to do it, but I want to be clear that I'm not opposed to a machine-readable list in addition to the human readable list as long as they're kept in sync. I don't see a reason we couldn't add that to this repo and close this discussion. PRs would be welcome.
add that to this repo
then, the final point would be to deceide on the repo(s) where that yaml index and generated markdown would be
the final point would be to deceide on the repo(s) where that yaml index and generated markdown would be
It would be in this repo and it would update the README.md. I want people adding a template to only have to do it in 1 repo - as someone who was in charge of that with the old setup, it's a pain tracking both of those PRs and not something I'm willing to continue doing.
i think instead of having it in readme, it would be better to have TEMPLATES_LIST.md (or just TEMPLATES.md mb) linked from a readme - it would help to keep readme much more laconic for reading, considering the number of currently supported templates
and yup, i also think having both in same repo would be easier, as it would not require setting up tokens for CI
it would be better to have TEMPLATES_LIST.md (or just TEMPLATES.md mb) linked from a readme
I have to agree here (though perhaps we can still find a slightly better name)... a separate file for a long list that never gets caught up in merge conflicts with the README is the way to go IMHO.
alright, then i'll try to implement the pipeline this weekend
Shouldn't we first agree what needs to go in that file?
The original list seems to have had:
- maintainer github username, link to maintainer
- nice name of template
- link to github repo for template
Do we need anything else? I think we'll (soon) need to know which versions of the spec a template supports - if this list is supposed to be canonical and useful for automated tooling... (not sure if we'd surface that in the markdown though)
for providing any other information which couldn't be extracted from template name or URL - the spec for templte's config.yml should be updated #3 (comment)
I'm all for adding a description field. #16
so instead of reading my motivation in my messages in related topic and arguing against them you decided to argue against what, as you can imagine, would be my motivation? this is just wrong at all levels
No. If I appears to be straw-manning you it was certainly not intentional. Moving this back on topic (I think)... if not please create a new issue regarding your specific concern (since you seem to say now it's not build all)... it's possible it's gotten lost in all the words and a single clear paragraph of your use case in a separate issue would provide enlightenment.
it described all along this thread, mainly in #3 (comment)
or, to put it differently, it's needed for same reason as repository index in linux repositories - it doesnt mean what you going to install all packages from them at once, but any data-store need to have an index. in case of schemes the index now is filetree. in case of templates the index is missing after forking/dropping it from spec.
Yes, I responded: #3 (comment)
I'm honestly very curious to see what you' build with this index once you have it back... :-)
I would also like to argue that we should keep this index out of the spec for now - it will let us iterate on it now and potentially formalize it later.
No argument from me on that point.
I think we're going to drop this for now - I would love to see a proposal around this, or some sort of "manager" which handles this, but I don't think we have any plans to work on it.
Hopefully a proposal would be able to answer the following questions:
- What happens when a template repo is down/gone?
- What happens when a template fails to compile?
- How will repo moves be tracked?
- What happens when a template needs a non-standard builder (such as a few which need to export binary files)?
Those are all issues we ran into in the past, and based on how the early builder specs were worded, it caused builds to fail for unrelated templates multiple times.