Remove all options from Homebrew/homebrew-core formulae
MikeMcQuaid opened this issue · 85 comments
Options in formulae don't produce a good user experience because they have to be built from source, we don't test them in CI and each combination of options provides a new chance for new failures to occur. We should seek to (eventually!) remove all options from formulae in Homebrew/homebrew-core in favour of enabling as much non-exclusive functionality as possible in a given formula for widely used options or encouraging the community to maintain their own custom options in a tap (e.g. https://github.com/denji/homebrew-nginx/blob/master/Formula/nginx-full.rb). As an absolute last resort if we need to depend on the same formula multiple times with different options (e.g.
#13133) we can consider vendoring formulae using resource blocks or even duplicating formulae.
This should be done in multiple phases:
- formulae that have build error issues created with particular options should have the options removed immediately (rather than a fix being merged)
- less popular formulae should have all their options removed
- popular formulae (e.g. https://formulae.brew.sh/analytics/install-on-request/90d/) should be prioritised for having their defaults adjusted (depending on analytics usage) to get the best balance of functionality without options.
I'm open to thoughts on the above and different approaches to solve the problem as well as additional tooling required in Homebrew/brew to accomplish this.
What about a more general approach akin to Debian's alternatives system? That could handle the issue with options as well as other things.
One thing that I've always missed in Homebrew was being able to pick an implementation of MPI: Open MPI vs Mpich. Currently when one installs Mpich, packages depending on MPI (and so on Open MPI in Homebrew's logic), such as Scalapack, have to be built manually, else Homebrew complains about missing dependencies.
Options in formulae don't produce a good user experience because they have to be built from source, we don't test them in CI and each combination of options provides a new chance for new failures to occur.
Those things do happen with options, but I don't think it follows that eliminating options provides a better user experience. A formula that lacks needed options means the user makes their own tap, in which case...it will be built from source, not tested by CI, and provides new chances for failure. I.e., those three things that an option-using user currently faces will still be faced just about as often once options are eliminated.
What about a more general approach akin to Debian's alternatives system? That could handle the issue with options as well as other things.
We can't really do this given our limited CI and options aren't a good approach for this. "Picking an implementation" isn't really something Debian supports for compile-time things; as far as I'm away that only applies for things that can be swapped out underneath (i.e. are API and ABI compatible) without an application change. This is theoretically possible as something we could do but mostly people are requesting the ability to swap out things that aren't necessarily compatible.
Those things do happen with options, but I don't think it follows that eliminating options provides a better user experience. A formula that lacks needed options means the user makes their own tap, in which case...it will be built from source, not tested by CI, and provides new chances for failure. I.e., those three things that an option-using user currently faces will still be faced just about as often once options are eliminated.
I think a less flexible Homebrew/homebrew-core that works for every user every time is superior to a more flexible Homebrew/homebrew-core that works for every user 90% of the time. That's a product/design difference of opinion I realise we don't share.
Essentially, though, you touch upon a good point: for things that have to be built from source or build using options there's no real difference between Homebrew/homebrew-core and a tap that doesn't provide bottles. As a result it's much, much easier for the community to step up there and provide solutions to these problems without adding the negative perception when Homebrew/homebrew-core doesn't work and the issues that result from hard to reproduce, option, from source builds.
I think a less flexible Homebrew/homebrew-core that works for every user every time is superior to a more flexible Homebrew/homebrew-core that works for every user 90% of the time. That's a product/design difference of opinion I realise we don't share.
I personally tend to agree with @chdiza. (Although I'm willing to concede that I may be in the minority here, and I may not have had the extensive experience that led @MikeMcQuaid to those conclusions.) However, if we go this route, I think we need to greatly improve our documentation for setting up dedicated taps, with all the bells and whistles like bottling etc. I was looking into this a few months ago, and there is no straightforward guide on how to accomplish this. I finally stumbled upon this repository, which seemed like the best example for me to follow. Better documentation for users/contributors of the bottling process would make this type of migration much less of a bitter pill for some (like me!) to swallow.
I think we need to greatly improve our documentation for setting up dedicated taps, with all the bells and whistles like bottling etc. I was looking into this a few months ago, and there is no straightforward guide on how to accomplish this.
The current documentation (e.g. https://docs.brew.sh/How-to-Create-and-Maintain-a-Tap and the brew tap-new command in the manpage) is sufficient for many people to setup taps. If you would like to extend that documentation: please feel free.
with all the bells and whistles like bottling etc.
This doesn't really make sense; options are never bottled anyway so to remove them doesn't imply we need to provide an ability for others to bottle them.
Better documentation for users/contributors of the bottling process would make this type of migration much less of a bitter pill for some (like me!) to swallow.
The issue isn't documentation, really, it's setting up free building of macOS packages (on multiple OS versions) with free uploads to a free CDN. I don't think that problem is ours to solve, really, but again: if you're interested in solving it please feel free.
For the record: I think the mass removal of options is patently ridiculous & achieves nothing except stymying debate about individual options because those PRs are too large to easily discuss just one or two elements.
I also think it is ridiculous to move ahead with this after 10 days of this issue being opened, when precisely two maintainers thus far have weighed in on the subject, and the core team as a whole was essentially rebuilt from the ground up less than one month ago.
Some of these options also have effectively zero maintenance burden, or have never been knowingly problematic in terms of either analytic-logged build failures or human reports, and yet get removed anyway. Other options were the product of days or weeks of discussions about how best to resolve other issues, like Perl binding versioning & incompatibility, and have been casually thrown out the window with zero debate.
I truly tip my hat to the "Ram the change through as rapidly as possible and hope most people, including maintainers, don't notice or object until it's too late" tactic.
Fair. I'm on the train and my internet connection is terrible, so I didn't do due diligence before I made that statement.
While I personally like options, and find that they add value to Homebrew for power users or devs who need a certain package built a certain way, I do see that there is a trade off and it's not a one sided issue. If it were put to a vote (I'm NOT saying the it necessarily should be...) I would vote to keep --devel and --HEAD options, and want to evaluate other packages on a case by case basis.
But anyway, I'd be curious to hear what other from @Homebrew/core think, if they have a strong opinion. There's a fairly good chance that they all agree with Mike which is why they didn't feel the need to comment here... 🤷♂️
On the whole I'm not a fan of options for the reasons cited by Mike. I do like --HEAD, because it can be useful when troubleshooting upstream bugs. I don't see much benefit in going to the effort of removing options that have caused no documented trouble, either in analytics or issues. I'm in favour of removing options that have documented trouble. I'm in favour of adding no new options to formulae.
mass removal of options
I'll be holding the open PR, and not opening new ones. I apologise if you felt those were rushed: it clearly was not my intention: I just got a little time to do cleaning-type stuff while I was sat at a conference, and took it. Given that: 1. the PR was open for more than a week, 2. it doesn't look much like a call for discussion, but a plan for action, and 3. no maintainer has objected, I figured it was OK to move forward.
I've tried to remove options from formulas with very few install counts, i.e. options removed should be lower than 0.5% in usage or lower than 5 in absolute install count (per 30 days).
As to my personal opinion: apart from a few formulas where they are "natural" (aspell, some video or audio players), options run counter to our main goal, i.e. “just working”, for the arguments given by Mike. If we want to keep many options, I'd like us to consider running all option combinations through CI (which limits their number) or officially declare them “semi-supported” (i.e., we remove them when they break, unless someone patches it).
I feel that keeping things lean and simple is a good thing. Also, existing documentation on maintaining a user tap is just fine.
What we could do better is advertise the user tap feature more often. As in, whenever options pop up as a topic in a PR.
I truly tip my hat to the "Ram the change through as rapidly as possible and hope most people, including maintainers, don't notice or object until it's too late" tactic.
I don’t think bristly accusations are warranted
@DomT4 I strongly agree with @zbeekman: this was not an acceptable way to refer to the work of other maintainers such as @fxcoudert who received a PR approval before merging based on an open "help wanted" issue.
I also think it is ridiculous to move ahead with this after 10 days of this issue being opened, when precisely two maintainers thus far have weighed in on the subject
There is a degree of obligation on us maintainers to follow these proposals and issue tickets and voice our opinions.
Again, I agree with @zbeekman: this issue was open for 10 days in which you have been active on this issue tracker and in Slack and you had no comments on it during that time. You've also (re)joined the team more recently than @fxcoudert; he has no obligation to ask you for permission to make a change like this.
I'll be holding from the open PRs. I apologise if you felt those were rushed: it clearly was not my intention: I just got a little time to do cleaning-type stuff while I was sat at a conference, and took it. Given that: 1. the PR was open for more than a week, 2. it doesn't look much like a call for discussion, but a plan for action, and 3. no maintainer has objected, I figured it was OK to move forward.
I've tried to remove options from formulas with very few install counts, i.e. options removed should be lower than 0.5% in usage or lower than 5 in absolute install count (per 30 days).
You have nothing to apologise for @fxcoudert. It's worth noting we've not been accepting options on new formula for a long time now. When we imported formulae from taps (many of which were in our top 1000 and sometimes top 100) we've removed all options in doing so. This has resulted in dramatically fewer build errors and those formulae being easier to support.
To be extra explicit: this has been an undocumented but planned policy for several years.
What we could do better is advertise the user tap feature more often. As in, whenever options pop up as a topic in a PR.
@claui I agree.
Options in formulae don't produce a good user experience because they have to be built from source, we don't test them in CI and each combination of options provides a new chance for new failures to occur.
@fxcoudert Is the only person in this thread to even offer a suggestion of how to resolve any of these issues (run them all through CI). If someone points out an engineering problem like this: suggest an alternative solution.
I would vote to keep --devel and --HEAD options, and want to evaluate other packages on a case by case basis.
To be clear on the scope of this PR: I do not consider --devel or --HEAD to be "options" as far as this PR is concerned so I'm not proposing their systematic removal.
I don't see much benefit in going to the effort of removing options that have caused no documented trouble, either in analytics or issues
@sjackman on this it's worth noting:
each combination of options provides a new chance for new failures to occur
so even if option A and option B may both work fine when both specified together they produce a failure. This produces a dilemma: does option A get removed, option B get removed or what? I'm just personally unconvinced that this is a problem worth us dealing with when taps can maintain optionful formulae as well as we can.
each combination of options provides a new chance for new failures to occur
Each new formula provides a new chance for failures to occur, and I don't see much objection to adding new formulas. Each formula version bump provides a new chance for failures to occur, but HB bumps versions all the time. So it can't just be that options will bring more failures or whatever, but that the cost isn't worth the benefit. I've seen no argument whatsoever for this last part. I've only seen a list of potential costs.
Some of these alleged costs really don't seem costly at all to me, and are unlikely to be seen as costly by an option-wanting user. Options won't have CI, but so what? None of Homebrew had CI for a long time, and while it's certainly better with CI, a CI-less Homebrew was much better than no Homebrew at all. Same for options. Options must be built from source, but so what? Having to build it from source is better than not having it at all.
But even if they really were costly, still what I'm seeing is:
"We won't have options, because that would suck for you: you'd have no CI and you'd have to build from source, and also the build might not work. So our advice is to go make your own tap, where you will build from source and not have CI, and where the build might not work."
That is so absurd that I think the CI and source-build stuff are just extras obscuring the real motivation, and the real underlying thing is "We won't have options, because that sucks for us: it will generate tickets. So our advice is to go make your own tap; and if your build fails you shouldn't make a ticket (except maybe when it seems clearly our fault)." This is also absurd, but in a totally different way.
Each new formula provides a new chance for failures to occur, and I don't see much objection to adding new formulas.
Each new formula can be bottled. Bottle failures are much rarer than build from source failures (which are rarer than non-CI-tested build-from-source failures e.g. options).
None of Homebrew had CI for a long time
Yes, and it sucked. I was a maintainer before we had CI and no-one else in this conversation was. Homebrew was worse quality, had a poorer user experience and many fewer users and contributors as a result. Relatedly, I want Homebrew to outlive my involvement in the project and part of that is defining the scope of the project in relation to the expectations of the users and, frankly, yes making the project easier for the maintainers to maintain and focusing on the stuff that only we can do in this tap and not the stuff that a copy-paste into another tap will do identically.
That is so absurd ... obscuring the real motivation ... also absurd
And at this point the technical discussion ends and the hyperbole begins. If you're going to speak in this way this conversation is going to get locked very quickly.
We won't have options, because that sucks for us: it will generate tickets. So our advice is to go make your own tap; and if your build fails you shouldn't make a ticket (except maybe when it seems clearly our fault).
Expectation management is no easy task. I’d never turn down anyone who’d ask for advice over a custom formula. I still believe drawing a line somewhere is a healthy thing to do. Even if it’s only to manage, quote unquote:
the negative perception when Homebrew/homebrew-core doesn't work
This is also absurd, but in a totally different way.
How so?
Again, I agree with @zbeekman: this issue was open for 10 days in which you have been active on this issue tracker and in Slack and you had no comments on it during that time. You've also (re)joined the team more recently than @fxcoudert; he has no obligation to ask you for permission to make a change like this.
I can only apologise for having to prioritise where I spend the time I have for Homebrew.
The vast majority of the time that I spent on Slack recently has been 1) having a really prolonged discussion with you post-lead maintainer stuff, 2) talking Claudia & Izaak through the pull mechanism, 3) trying to work through the Perl changes in Mojave. Feel free to let me know which of those I should've dropped to free up additional time.
I took a look at this issue, decided it would be quite a complex discussion and dared to assume we wouldn't be bombing ahead with it within a fortnight & I'd have time to generate a discussion point that wasn't overly-simplistic. If feedback is that urgent maybe point out in Slack that you'd like opinions sooner rather than later, as I did repeatedly with things like the libclang PR. maintainer feedback as a label does not imply "the second I get one piece of feedback we're gonna move on this".
As for your little personal ding at the end, yes, as someone who at times single-handedly maintained most of core for an extended period in the past & has the most commits to core of any maintainer today, I generally am a little surprised when people don't at least ping me for an opinion on things that represent a major change in core, which isn't to say one has to agree with that given opinion.
======
@fxcoudert - Outside of everything else, I apologise for my overly personal tone in my initial comment. The rushed nature of the changes bothered me but I didn't intend to swing at you personally.
I took a look at this issue, decided it would be quite a complex discussion and dared to assume we wouldn't be bombing ahead with it within a fortnight & I'd have time to generate a discussion point that wasn't overly-simplistic.
@DomT4 It sounds like you as a reviewer were expecting people to wait for a specific amount of time for objections. We may all have different opinions about what is and isn't controversial. I also understand not having enough time to fully express my thoughts about something, and so I often don't say anything. But if one has some really strong objections to the PR, maybe it's better to say "I have concerns about this, and I don't have time to express it right now, but I would prefer if we wait before merging this"?
But if one has some really strong objections to the PR, maybe it's better to say "I have concerns about this, and I don't have time to express it right now, but I would prefer if we wait before merging this"?
@scpeters I could not agree more strongly.
I can only apologise for having to prioritise where I spend the time I have for Homebrew.
@DomT4 Every maintainer and contributor working on Homebrew has limited time to spend on it. That goes without saying. If replying to this issue was not a priority for you thats completely up to you but, as @scpeters pointed out above, it's not reasonable to expect others to wait for your input when you haven't requested we do so and, as I pointed out above, this is the documented continuation of a policy that predates you rejoining the project.
someone who at times single-handedly maintained most of core for an extended period in the past
- You rejoined as a maintainer a month ago after leaving for 21 months (in which I periodically tried to convince you to come back). Previously you were a maintainer for 18 months.
- @fxcoudert has also been a maintainer for 19 months in total and obviously has a longer continuous streak and recent streak.
- I've been a maintainer for 107 months.
As a result, if @fxcoudert and I reach a documented decision in a public issue it's reasonable for him to move forward with that decision after a week.
has the most commits to core of any maintainer today
Today: yes. When this issue was opened: no. We're really grateful for your contributions to Homebrew/homebrew-core but a number of commits does not grant you special privileges as when you stepped down your commits needed to be merged by other maintainers.
Please give i18n support for git to us that whose native language is not English.
git's i18n is so well, and that helps me a lot.
If you want remove the option --with-gettext, why not build with i18n by default like git's official website?
@moonfruit we could build git with gettext by default. It would match our existing practice of shipping feature-full bottles that benefit most users.
Yes, and it sucked.
Perhaps it sucked for maintainers. For users, it was much, much better than nothing.
The removal of options benefits exactly zero users (and harms some of them), but somehow is being pitched as being to the benefit of users.
We're trying to hear everyone, but at some point, actions speak louder than words. It is very easy for users to maintain a tap, that has exactly the same functionality as our existing options: no "seal of approval" from Homebrew core, no bottles (by default; tap maintainers can add bottles, but we agree it is significantly more difficult). That there is so much complaining, but (to my knowledge) no taps being maintained for the formulas affected, speak volumes. To me, it says "yes, options were nice to have for a very small number of users, but people are not affected very negatively”.
I should add that some projects (ethereum, neovim, etc.) use external taps for this reason: to maintain many options in their formulas. It does work very well for them, so we know it is a viable option to offer.
We do have, at some points, to make decisions. And no decision can please everyone. We take into account your feedback, as well as that of others, and our perspective as well… and make the best decisions we can. We are not opposed to revising previous decisions, and admit we make mistakes (such as our previous handling of python2 and python3, to cite just one example). However, at some point, arguing endlessly without new arguments is not worth the bits.
On the specific issue of git: we are moving forward on a compromise that I believe is better for all users. See #32008
External taps are indeed viable. I've never denied this.
What I deny is that the removal of options benefits users. So the reason offered for removing options is entirely meritless.
And, "well, it may not benefit users, but it also doesn't irreparably harm them either" is not a reason for removing options.
On the specific issue of git: we are moving forward on a compromise that I believe is better for all users.
It is better for some users. The rest of us now have to either make a custom formula for git that excises unwanted things (as it happens, I did this long ago), or we now have to brew gettext and pcre even though we don't want or need them.
Slowly Homebrew has been turning into "Macports-but-in-Ruby". The world was not clamoring for such a thing when Max invented Homebrew. (I thought one of its points was specifically to not be like MacPorts-from-back-then and saddle everyone with deps galore.)
The rest of us now have to either make a custom formula for git that excises unwanted things (as it happens, I did this long ago)
If you already have made a custom git formula tailored to your needs, how do you expect a hypothetical removal of options to affect you?
or we now have to brew gettext and pcre
“We have to brew” sounds as if that involved effort or inconvenience from the user’s side.
even though we don't want or need them.
[…]
saddle everyone with deps galore.
I realize you’re not happy, and I feel sorry about that. But then again: you’re an advanced Homebrew user. You appear to have somewhat strong opinions about how exactly your system should look like. You know how to write and customize a formula. What’s keeping you from doing exactly that?
Slowly Homebrew has been turning into "Macports-but-in-Ruby". The world was not clamoring for such a thing when Max invented Homebrew. (I thought one of its points was specifically to not be like MacPorts-from-back-then and saddle everyone with deps galore.)
Homebrew is still very different to MacPorts in many approaches but you may be right in saying the two projects are converging in their approaches. The main thing, which we've talked about before, is that Homebrew has moved from being a from-source system-optimised package manager to being a binary package manager. Looking at the analytics data it's worth noting: most Homebrew users use it this way.
I realize you’re not happy, and I feel sorry about that. But then again: you’re an advanced Homebrew user. You appear to have somewhat strong opinions about how exactly your system should look like. You know how to write and customize a formula. What’s keeping you from doing exactly that?
Indeed, I agree. Also, and I hate to say this because I genuinely value much of your input and your bug reports, but it's possible your use-case is no longer the use-case the Homebrew project is primarily optimising for (whereas I think when the project was created you're right in saying it was optimised for your use-case).
What I do want to do, though, is make it as easy as possible to create and maintain your own tap. brew extract is part of this, brew tap-new is another and
Homebrew/brew#4767 should make it so that tap and from-source formulae are less likely to be silently broken.
If you already have made a custom git formula tailored to your needs, how do you expect a
hypothetical removal of options to affect you?
I don't. It affects others, who've not already taken the steps of modifying a formula, learning enough git to maintain a tap (or trial-and-erroring until one figures out how to have a non-git-repo tap), etc., etc. It so happens that I already invested all of that long ago when it was only certain options, and not all options, that were on the chopping block. But it was a big investment. There's no reason to make others do this.
“We have to brew” sounds as if that involved effort or inconvenience from the user’s side.
It does. (Maybe e.g., I can't build foo now because my connection is down and foo has an inexplicable hard dep on bar, which is not yet downloaded. Maybe my connection is not down but slow, etc. Maybe I like building from source and don't want to spend time building stuff I neither want nor need.) It's obviously not a massive or insurmountable inconvenience, but it's darn annoying. It's what made MacPorts so annoying back in the day. Meanwhile, the corresponding options don't hurt anything.
It affects others, who've not already taken the steps of modifying a formula, learning enough git to maintain a tap
But it was a big investment.
Good point. On the other hand, GitHub has become much more user-friendly, especially for beginners. In 2018, there’s absolutely nothing stopping you from creating a repo on GitHub with zero Git skills.
In 2018, there’s absolutely nothing stopping you from creating a repo on GitHub with zero Git skills.
But a lot stopping you from doing anything with it sans git skills. :) (I include using a git-controlling GUI under "git skills", since you have to know the concepts in order to properly use the GUI; it's just a shortcut for the CLI.)
Side point: It used to be that you didn't need to really have any clue about git in order to use HB. Git is used under the hood in installation and updating, but managing packages didn't require any git knowledge except in rare and complicated cases. (I know, because I managed packages for years without any git knowledge; even now I can barely make a PR without poking my eye out.) This was as it should be. But now git knowledge is creeping in. You need it for maintaining taps, and soon you'll need it if the formula in core doesn't expose an option that you need that's actually in the ./configure script. It was almost the case recently that reverting to an old version of a package would require git knowledge (had the automatic deletion of outdated kegs come to pass). This kind of conflicts with the "HB is for everyone, even if you don't know what $PATH is" vibe that's been going around.
But a lot stopping you from doing anything with it sans git skills.
you have to know the concepts in order to properly use the GUI; it's just a shortcut for the CLI.
Again, this is no longer the case. There’s a large button on your repo’s main page that says Create new file. It literally can’t get any less related to Git concepts.
-
Create repository.
-
Create new file.
That’s all there is to maintaining your own formula.
That’s all there is to maintaining your own formula.
If you ever change that file, then ...
If you ever change that file, then ...
You can do that through the github UI. Gitlab even has a web IDE that you could use
And then if you want to save it, you are forced by github to make a choice about which branch to "save" it to, and maybe wonder whether you should create a different branch, and maybe even whether committing is the same as saving (if you really don't know anything about git), which means you have to know some git concepts. It's possible to bluff one's way through this, but it is not just as easy as using your own editor to save a file.
Anyway, this is all about something I labeled a "side note".
I'm late to this conversation and jumping in mid-conversation so apologies if my question has been covered...
I know homebrew has been trending this way for a few years now for good reason. I'm not a fan of this approach at the user level, but understand why it's happening. I really wish there were other alternatives. At this point, it's very much not a "homebrew".
So if some formulas are now having options removed and the onus placed on user taps for custom options, what's the guidance to avoid confusion? Would there be two formulae with the same name: one stripped of options that has a matching bottle in homebrew and a secondary with configurable options and no bottle in one or multiple user taps?
A formula I use, but apparently not popular enough, had it's options stripped. I can add it to my tap if necessary to mimic how it used to work. For someone else that wants to use the formula, how would they know which one to use?
For someone else that wants to use the formula, how would they know which one to use?
They can either Google for the formula in your tap or create their own.
For someone else that wants to use the formula, how would they know which one to use?
They can either Google for the formula in your tap or create their own.
Let me ask my question differently. Assume by pushing options to user taps for a given formula, homebrew will still keep their version meaning a minimum of 2 formulae and now a decision point for the user.
How would a user know which formula to use: the "official" homebrew version that has a bottle but no configurable options or the "unofficial" version in someone's tap that allows for configurable options.
Here's where I'm coming from...
- pushing this to user taps may (probably) lead to a lack of rigor and standardization that currently exists in homebrew
- anytime you make a user choose which formula to use you're now placing the onus on them and will lead to fragmenting the market for that formula
Again, I acknowledge you lot do the heavy lifting and much of the work and have your reasons. If I understand your answer correctly, you're already adding a step for the user to download a formula vs opening a prompt and brew install formula.
I assume most users will use what they perceive to be the official formula, but official is simply a lowest-common-denominator solution (my opinion).
Is there a way to have a bottle only version but at the very least tell users who are installing that formula that other user maintained formulas exist? I get the infrastructure isn't yet in place to do this yet so it's just an idea.
Most if not all of the options we're removing are used by less than 1% of our users (and many times, less than 0.1%). So removing options does not change much for the user: they will brew install foo, and 99% of the time, be happy with it. In 1% of cases, they will ask themselves: “where is power feature X that I would really like?”. If so, they can google for it, find the tap where it's maintained, and install it.
But for 99% of users there will be actually be fewer choices to make than before :) because we don't present them with options that most of them are not using anyway.
Honest question. For lesser used formula, are 99% of their users fine w/ default options or bottles?
Maybe at the aggregate level this only impacts <1% of all downloads, but I'm curious if that's factual for formulas that aren't as popular. My guess is it would be higher.
Specific to my original question, I came looking for guidance, but it doesn't appear that there is yet a best practice. So simply create whatever formulas that I personally want to configure in my own tap and try to keep it within homebrew guidance.
Specific to my original question, I came looking for guidance, but it doesn't appear that there is yet a best practice.
I agree!
Whenever I find a formula wanting an option, I’d copy/paste it to my personal tap and then move on. I believe this is less overall work than googling for a random third-party tap. I also don’t believe the formula market will become fragmented. The core tap is pre-installed, and thus much more visible than any custom tap ever will.
... at the risk of alienating users who crave a bit more control and who along w/ the maintainers helped homebrew get to where its at.
The direction offered a few years back as to where homebrew is heading - again, acknowledging the there are valid concerns - has negatively impacted my contribution to the community during that time period. I'm genuinely interested in a go forward solution that brings more people to the community than alienates the very people that can help.
One last question, since these formulae are not nearly popular enough, why have them in homebrew moving forward rather than direct people to a user tap?
For example, I care deeply about a lesser known formula, xplanet, and have been active in its edits and those of its dependencies or formulae that depend on it like xplanetfx. xplanet had it's options removed. If I'm going to recreate it in my own tap, why stop there? Shouldn't I also move other dependencies like potentially cspice that are also not that heavily used? How to know which formulae maintainers care enough about vs lesser used ones you don't want to deal with?
... at the risk of alienating users who crave a bit more control and who along w/ the maintainers helped homebrew get to where its at.
@blogabe We appreciate every single person who has helped make Homebrew what it is today, and I believe their voices deserve to be heard.
It’s not easy to make everyone happy though. Please help me understand your concerns a little better. As a power user, where exactly do you feel is the lack of control, given you have the possibility of maintaining select formulae in your own tap, get to decide on every single bit of them, and have your tap seamlessly integrate into Homebrew?
I’d really love to understand your point of view better. In my books, maintaining my own formula is the most control I feel I can possibly get. How does that make you feel alienated?
I guess it comes down to lesser than formalue will now be more difficult to find. Why bring google into this.
I'd be more open to these changes If we can facilitate this in homebrew. Maybe taps that want to be visible through brew search can link back or register to homebrew (maybe there's a way to do this now and I'm just unaware).
Idealistically, I'm not a fan of these changes because I believe it's fundamentally different from homebrew's original mission. However, its the maintainers that have to deal with these issues on a daily basis . I see the need. But I simply don't like it :|
Firstly I'd like to thank you for your polite, kind and considerate tone @blogabe. While we disagree I'm glad we can have a reasonable discussion about this.
How to know which formulae maintainers care enough about vs lesser used ones you don't want to deal with?
In general you should expect that eventually all formula options may be removed.
I guess it comes down to lesser than formalue will now be more difficult to find. Why bring google into this.
I'd be more open to these changes If we can facilitate this in homebrew. Maybe taps that want to be visible through
brew searchcan link back or register to homebrew (maybe there's a way to do this now and I'm just unaware).
We've previously tried that approach and it didn't work well. People expect things that show up in brew search, options, devel etc. to be supported by the Homebrew maintainers and, well, work. This is often not the case and we don't have the resources to make it not the case (the project being volunteer run, after all). If there was a way to allow users to add more things to the search not by default (HOMEBREW_SEARCH_UNSUPPORTED_TAPS) that did a GitHub Search and showed (unsupported) that might work.
Idealistically, I'm not a fan of these changes because I believe it's fundamentally different from homebrew's original mission. However, its the maintainers that have to deal with these issues on a daily basis . I see the need. But I simply don't like it :|
Thanks for the empathy and understanding. I was contributing to Homebrew a few months after the project started and a maintainer about 6 months after it started and I don't agree that this differs from our original mission. The big difference is that back then we didn't have CI or binary packages and most things that broke were handled by the users (who were generally relatively experienced developers). This has changed as Homebrew has grown such that to accomplish the same goal for our users we need to slim down what we attempt to support due to what we're able to support.
I'm a fan of making brew search discover content in taps, particularly when nothing is found in Core. It can be accompanied by a warning messages that community taps are not supported. The user chooses to tap the tap. If brew install fails for a formula in the tap, it already displays a message not to contact Homebrew about the issue.
EDITS: punctuation, grammar, clarity... I still am not about to win any writing awards though 🙃
Firstly I'd like to thank you for your polite, kind and considerate tone @blogabe. While we disagree I'm glad we can have a reasonable discussion about this.
💯
I'm glad to see that this discussion continues in a respectful manner. 😄
I'm a fan of making
brew searchdiscover content in taps, particularly when nothing is found in Core.
I am concerned about driving users to private taps. This has the potential to fracture the software eco-system. Encouraging the widespread adoption of taps may introduce incompatibilities with other packages, or a decline in user experience due to taps that are poorly maintained or malicious. See, for example, recent developments with Heroku. Furthermore, it may make it less likely that important patches in formula make their way into core, and less likely that important patches in the package itself makes it upstream.
I've expressed my opinion before that I do like some of the flexibility that options provide, and, if it were up to me alone, I would not completely remove them. However, as I have not studied the analytics data, I am not in a position to demand (or politely ask) that we hold off on completely abolishing all options. A data based policy, based on relative and/or absolute thresholds for which options get removed could help reach a compromise and end up providing the most benefit to the most users. It also might make it seem less arbitrary.
No matter what, we do need to reduce the complexity of maintenance as the project grows, as adoption grows and as more formula are added. Removing options is clearly advancing this goal. And some/most options should probably be removed.
However, as I have not studied the analytics data, I am not in a position to demand (or politely ask) that we hold off on completely abolishing all options. A data based policy, based on relative and/or absolute thresholds for which options get removed could help reach a compromise and end up providing the most benefit to the most users. It also might make it seem less arbitrary.
To be clear: options are not being removed because they are used or unused but they are being removed because we do not test options with CI or build binary packages for them and as a result they are regularly broken. Analytics are useful in figuring out what the default configuration should be.
If
brew installfails for a formula in the tap, it already displays a message not to contact Homebrew about the issue.
Worth noting here: but yet many of them still submit issues to us anyway. Unfortunately this is why I think any search behaviour shouldn't be enabled by default.
Random thoughts that may or may not help the conversation...
-
There are plenty of COTS software that have three tiers of support: 1) official from vendor w/ full testing/support 2) technically works, but supported through 3rd party or open source community and 3) technically may work but you're on your own. I can see homebrew going down this path (even if it failed before, doesn't mean it will fail in the future).
-
With respect to homebrew still maintaining the original ideals, to me it's the difference between brewing your own beer or buying it at the grocery store -- I went far and deep to find that analogy :) What attracted me to homebrew in the first place was the ability to brew myself, on my machine, with my configuration. Going to bottles, is well, ... going to bottles :)
-
With respect to the mechanics of how this all works, I don't yet see the same end result the maintainers do as to how this reduces the complexity. I think it has potential to make things much more chaotic... at least for user taps which means less standardization then fragmentation and alienation. Again, maybe because I simply can't visualize how this looks given the rate of change and discussion in this project. How will dependencies by handled? If there are multiple versions of a dependent formula how can the user through the command line or the requesting formula choose the one it wants?
Am I better served off w/ having all dependent formulae hosted in my own taps alongside the formula I am now required to host?
Using my above example, there is a program I host in my tap, xplanetFX. Used to be part of core but removed due to #13356, #14019, and Homebrew/brew#2482. It has a variety of dependents including xplanet (which itself has seldom used dependencies like cspice) and pygtk with the libglade option. As I'd copy xplanet to my tap as well, I'd change the dependency in xplanetFX to reference my xplanet formula, correct? But what if someone wants the xplanet bottle through core instead? I get xplanet and FX are seldom used, but I'm using it as an example. Maybe other formulae have similar issues, but are more popular.
- I echo @DomT4 original sentiment... maybe this has been building for a while, but this specific change seems to have been intro'ed, discussed, and decided upon pretty quickly. I think in the future if want more discussion or engagement, certain changes like these that impact many formulae, and their dependencies (as described above), need more wide spread conversation and longer baking period. For example, maybe send out a tweet directed people to the conversation at hand. Between the multitude of issues to look through, the forum site, Twitter, ... I can barely keep up as a volunteer user.
Anyway, thanks for the opportunity to discuss.
There are plenty of COTS software that have three tiers of support: 1) official from vendor w/ full testing/support 2) technically works, but supported through 3rd party or open source community and 3) technically may work but you're on your own. I can see homebrew going down this path (even if it failed before, doesn't mean it will fail in the future).
The main issue here is that we get a bunch of issues filed for 2) and 3) regardless whereas people don't tend to ask COTS vendors for support in those cases (or their paid support staff can ignore or fob them off).
- What attracted me to homebrew in the first place was the ability to brew myself, on my machine, with my configuration. Going to bottles, is well, ... going to bottles :)
Indeed, we've been a primarily binary package manager for a while and this makes us more reliable. There are other options if you'd rather use a from-source package manager.
maybe this has been building for a while, but this specific change seems to have been intro'ed, discussed, and decided upon pretty quickly.
This has been literally building for years. Unfortunately we've had too much of the discussion here in private locations (e.g. the Homebrew maintainers' Slack) and I screwed up there: I'm sorry.
Between the multitude of issues to look through, the forum site, Twitter, ... I can barely keep up as a volunteer user.
I appreciate your efforts. That said, the direction of the project is decided by the maintainers of the project, ultimately, as they are the people doing the bulk of the day-to-day work. I'm happy to take feedback and ideas on how we can make it easier for others to maintain their own stuff however they'd like, though.
Anyway, thanks for the opportunity to discuss.
You're very welcome ❤️
Was it so hard to leave a comment in Git commit comment WHY you remove options? At least, with ref to this thread. I was made to create an issue and my compatriot referred me to this thread. Wow, sudden decision. Now, instead of just installing the package I need (option I contributed before) I need to learn how to make a tap, and so on. It makes a problem to me right now, I have no time to do this right now, so my own Homebrew experience right now become worser. I can understand this motivation about removing, and at the same time I cannot. Now people should have tons of unmaintenanced taps with options formulas. I don’t know much about taps, can it have parent? I.e. when “core” package bump version, will my tap bump it automatically? It is the only real important thing that bother me: will the package be in sync with core automatically or not, or then I do not see any profit in Homebrew, if I need maintenance any non-standard config manually. Looks like Gentoo vs Debian, huh )
Was it so hard to leave a comment in Git commit comment WHY you remove options?
@m-emelchenkov It requires extra effort and this is an open issue of a relatively small number in this repository so it is not necessary.
Wow, sudden decision.
This issue was created in August and this process of removing options has been ongoing for almost a year so: no.
I don’t know much about taps, can it have parent? I.e. when “core” package bump version, will my tap bump it automatically?
It will not.
then I do not see any profit in Homebrew, if I need maintenance any non-standard config manually. Looks like Gentoo vs Debian, huh
That's a good comparison because we are aiming to move from being more like Gentoo to more like Debian. Debian does not allow you to install arbitrary packages with arbitrary options. The goal is to have it for Homebrew/homebrew-core so everything that users install is tested in our CI system and we are comfortable that it works.
What is the stance against formulae that use options that don’t impact the build, as well the ones no other formula relies on?
I'm asking this in the context of kubectx.rb, the --with-short-names flag was used to determine only the symlink name of the bin.
@ahmetb All options will be removed. They still "impact the build" as they force a build from source for end-users and aren't run by CI.
Options in formulae don't produce a good user experience
@MikeMcQuaid We can have both a good UX and optional build options together. We can provide more sane default options for users who install the package without any options.
I can only think of one reason why all options should be removed, which is to minimize the number of issues reported by users.
We can have both a good UX and optional build options together.
@danieljl You've stated that we can but not how. How can we have binary packages for all option combinations? How can we have enough CI resources to build all option combinations? Who is going to do the necessary work to get tall that setup?
We can provide more sane default options for users who install the package without any options.
This is what is happening.
I can only think of one reason why all options should be removed, which is to minimize the number of issues reported by users.
Not just issues reported but issues experienced. As a very highly used open source project run entirely by volunteers: yes, this is a worthy goal by itself.
We're getting closer! The goal is now to remove all options (including :optional and :recommended) from Homebrew/homebrew-core packages for Homebrew 2.0.0 (expected Q1 2019).
Hi,
I understand your reasons to do this, but there are some formulae which are even harder to use if you don't have options to modify them, for example yarn. Its only option was --without-node which was very helpful for people who are using nvm. According to https://formulae.brew.sh/analytics/install-on-request/90d/, 1.85% of requests are yarn and 0.47% are nvm. It doesn't mean that 25% of people who use yarn are using nvm, but a lot of them is doing that. And as far as I know, the option only removed a dependency which isn't required to successfully install yarn. It looks like you're not taking any arguments why NOT to remove all options from all formulae (which is the reason I'm moving to macports), but there are also formulae whose installation can't be broken by some options. Just think about keeping these options.
I understand your reasons to do this, but there are some formulae which are even harder to use if you don't have options to modify them, for example
yarn. Its only option was--without-nodewhich was very helpful for people who are using nvm.
@MekliCZ This applies to many things installed by Homebrew: if /usr/local/bin is first in your path it will override the binaries elsewhere. I'm not sure I understand why putting nvm's nodes before /usr/local/bin in your PATH doesn't fix this?
In short: I'm not claiming that removing options will not require workflow changes for some users. Where we/I can I want to make those workflow changes as obvious an easy as possible.
And as far as I know, the option only removed a dependency which isn't required to successfully install yarn.
It is needed to run or test yarn, though, and it makes us a pretty crappy package manager if we don't install things you need to run the application you install...
It looks like you're not taking any arguments why NOT to remove all options from all formulae
This thread is full of arguments and I unlocked it. What I've said to many people (though) is that they needed to provide solutions to the problems which motivated this change. No-one did provide solutions beyond essentially "just ignore those support requests" so: here we are.
which is the reason I'm moving to macports
I hope it's a better fit for you. If this is your main issue, though, it seems strange. MacPorts also installs node when you install yarn and doesn't provide a variant for this. It's been a while since I used MacPorts so I could be missing something they offer that we don't, here.
@MikeMcQuaid can't speak for the overall community of course, but a challenge with coming up with ideas that fit the maintainers requirements is that that outside the "lessen the support issues" request, I'm unsure what other constraints we should abide by. I'm using the word 'constraints' specifically to imply maintaining some level of playing well with one another versus going off and doing something completely different.
There are enough people impacting by these changes that it may make sense to work with us and provide some guidance on how we can continue using homebrew and have it support our needs. Creating an independent tap is one of them, and I do not see that as a problem.
What I do see as a problem is now having potentially multiple users creating a tap for a particular formula, say imagemagick, to support options. I would think this is a popular enough formula where we'de want to centralize the hosting rather than have multiple options.
Furthermore, using Google to find imagemagick places an unnecessary burden on the user and may come up with varying results.
This project has started to potentially reach a happy medium: https://github.com/portage-brew
I think a centralized, rather than decentralized, approach to a tap allowing for formula options is a way to go.
Ideally, updates to formulae would be made in one place and carried forward, but I'm unsure how realistic that is. I can say from my own personal experience that as I know host formula w/ options on my own tap, updating the official formula on homebrew is a lesser priority, e.g., adding a revision or bumping the version.
How do you or other maintainers suggest we work together to address? This isn't about maintainers vs power users... it should be about the the overall user community, those that are fine with bottles and those that like to tinker.
Creating an independent tap is one of them, and I do not see that as a problem.
Yes, this is the expected model and we want to make that easier.
I would think this is a popular enough formula where we'de want to centralize the hosting rather than have multiple options.
I agree that discovery is a tricky part. The balance is on discovery vs. expectation that things work by default/are supported by the Homebrew maintainers. Perhaps expanding on https://docs.brew.sh/Interesting-Taps-and-Forks would be a start for this? I don't want us to go back to when brew search searches a bunch of community taps because we don't want to be seen to recommend things that we haven't reviewed.
This project has started to potentially reach a happy medium: https://github.com/portage-brew
I'm happy to see projects like this emerge (pun intended).
Ideally, updates to formulae would be made in one place and carried forward, but I'm unsure how realistic that is.
Unfortunately it's not realistic to expect that updates will happen that won't break the options. Most options require passing something to configure or otherwise modifying the install method. I've suggested before there may be a way to do this (subclass the Formula and override install and options but leave url etc. the same) and I'm open to PRs that make that process easier. Even then, though, I'm unconvinced it will be easier than copying-pasting a formula into a tap.
How do you or other maintainers suggest we work together to address?
Beyond what I've said: I don't, I'm afraid. I'm happy to offer personal opinions but kind of the point of this change is that Homebrew maintainers are no longer responsible for how to go forward with this.
it should be about the the overall user community, those that are fine with bottles and those that like to tinker.
I'm afraid I disagree. There's some users whose desire to tinker causes a disproportionate support burden. They want to fiddle with things until they break, file an issue (or even PR) and then a bunch of time is wasted discussing a self-created issue.
It's not unusual for software that is popular with "tinkerers" to outgrow them as their needs and desires for the project become outliers. This is what has happened with Homebrew: a niche, from-source package manager has become a binary package manager relied on by millions of people.
The major issue we run into taps is that there is no way to give a tap preference over homebrew-core when it comes to dependencies. Even when the tap is pinned. This makes the situation on our build server very complicated.
For example in my case I need a version of poppler which does not depend on nss or qt. Because these have become hard dependencies in core, I provide a custom poppler in my tap.
However when brew installs poppler as a dependency of a core package (for example gdal in my case) it still installs the core version, not the one from the (pinned) tap. This differs from pinning in Debian, where a package from the pinned repo can be a drop-in replacement for the mainstream package.
This may seem small, but it makes the situation unmanageable. I need a few dozen modified formulas in my tap which are built with different options than core. This I can do, however brew keeps running into conflicts because it has already installed the core version of the same formula as a dependency.
I think if you expect people to run taps for customized formulae, brew should allow taps that provide replacements for core formula, and which get preferred even when installed as a dependency.
The major issue we run into taps is that there is no way to give a tap preference over homebrew-core when it comes to dependencies. Even when the tap is pinned. This makes the situation on our build server very complicated.
This is intentional. We do not want to have random dependencies in our tree swapped out with versions that may not work.
For example in my case I need a version of
popplerwhich does not depend onnssorqt. Because these have become hard dependencies in core, I provide a custompopplerin my tap.
To illustrate my example: if a package in Homebrew/homebrew-core relies on the nss or qt functionality: the custom poppler in your tap will break that package.
This I can do, however brew keeps running into conflicts because it has already installed the core version of the same formula as a dependency.
I've elaborated a bit on this here: Homebrew/brew#5615 (comment). I'm open to improving the docs to make this (i.e. use a different formula name) more obvious and well-documented.
So now, instead of copy-pasting homebrew options from Google, I need to use brew edit all the time when one single option would have solved my problem.
Even worse, editing formulas on my own is no better than the pre-homebrew age, when I read docs, carefully set up compiling options, and fail, and try again.
Popular Homebrew options are at lease given a chance to be tested and solved by the whole community. Without options, installing packages are only getting harder.
Please don't just remove all the options. Rather split the formulas into two taps, one ready-to-use out-of-the box (core), one with options that may not be tested fully, but can be improved together by the community.
So now, instead of copy-pasting homebrew options from Google, I need to use brew edit all the time when one single option would have solved my problem.
The sites you're Googling will be updated with correct instructions.
Popular Homebrew options are at lease given a chance to be tested and solved by the whole community.
They aren't tested before updates: that's the reason for their removal.
Rather split the formulas into two taps, one ready-to-use out-of-the box (core), one with options that may not be tested fully, but can be improved together by the community.
This is what's happening except the latter is being maintained by the community outside of the Homebrew organisation.
This may seem small, but it makes the situation unmanageable. I need a few dozen modified formulas in my tap which are built with different options than core. This I can do, however brew keeps running into conflicts because it has already installed the core version of the same formula as a dependency.
@jeroen Does Homebrew still give you conflicts if you use non-conflicting formula names in your tap (such as jeroen/core/custom-poppler), and control all the linkage manually (for now) via brew link/brew unlink?
@jeroen I feel your pain on this. A solution I use is to put everything you need (even "mere" dependencies) in your own tap. Then just copy the HB formulas---even the ones you don't actually have to adjust the flags for---into your own tap, modifying the "depends_on" part as necessary to point explicitly to your own tap. E.g., put a formula for gdal in your own tap, and change it to say depends_on "myuser/mytap/poppler" instead of depends_on "poppler".
This means you won't get bottles, but that just goes to show why build-from-source package managers are Good and Necessary.
I have about 60 kegs installed, and now none of them are from the core tap. They are from my tap, which carries several duplicates of core formulas (except for their dependency references are changed to point at my tap). This way, I will never have the rug pulled out from under me by a sudden HB decision to mess with deps or flags.
brew should allow taps that provide replacements for core formula, and which get preferred even when installed as a dependency.
It should, but the above describes a way to ape this. Having to do brew install myuser/mytap/foo instead of brew install foo was mildly annoying, but e.g. I made an external command that will auto-insert the tapname before any formulas passed.
Even worse, editing formulas on my own is no better than the pre-homebrew age, when I read docs, carefully set up compiling options, and fail, and try again.
@bumfo Would it help if Homebrew allowed you to write custom formula extensions, or some kind of formula patch, as @saagarjha suggested in a recent forum thread?
Here’s the relevant quote from this thread that outlines the suggested feature:
The benefits I get is that I the basic auto-updating portion is done by Homebrew itself, but whenever I build I run the “patch” (which I am imagining is something like me selectively copying the options I care about from an old commit to homebrew/core and telling Homebrew “here, apply this”) to customize it for my environment (transparently; the “patch” is applied each time a new update comes and brew update is run).
I have about 60 kegs installed, and now none of them are from the core tap. They are from my tap, which carries several duplicates of core formulas (except for their dependency references are changed to point at my tap).
@chdiza Such determination! Given that you’re already going that route: have you looked at https://github.com/portage-brew yet?
(Not being snarky or something, I 100% mean it; I’ve only glanced at portage-brew briefly but it looks like a promising fork to me.)
[…] I made an external command that will auto-insert the tapname before any formulas passed.
That is actually a really cool idea.
@jeroen I feel your pain on this. A solution I use is to put everything you need (even "mere" dependencies) in your own tap. Then just copy the HB formulas---even the ones you don't actually have to adjust the flags for---into your own tap, modifying the "depends_on" part as necessary to point explicitly to your own tap. E.g., put a formula for
gdalin your own tap, and change it to saydepends_on "myuser/mytap/poppler"instead ofdepends_on "poppler".
This is what I've debated doing as well. I've resisted for the obvious reason that it places an undue amount of maintenance on our personal user taps and why a centralized tap can help.
One formula may have many - 20, 30, 40, ... - dependencies that I need to also manage and update. It's a burden I'm not stoked on taking on. Hopefully if portage-brew gains traction amongst the community then this should not be as much of an issue.
This is what I've debated doing as well. I've resisted for the obvious reason that it places an undue amount of maintenance on our personal user taps and why a centralized tap can help.
I agree that centralising effort may be useful but disagree that this needs to be in the Homebrew organisation to be successful.
@MikeMcQuaid Don't disagree with you on this.
I think it may be easier and radiate a more inclusive feel for the end users, but it can work outside of Homebrew as well assuming enough power users the changes impact are willing to join and build up a critical mass.
We're good :)
My ignorance is showing, but is it possible to subscribe to a formula in core so that we can receive an alert if/when it undergoes an edit?
Nope, sorry. Someone could build it on to of GitHub's APIs, though.
This is intentional. We do not want to have random dependencies in our tree swapped out with versions that may not work.
I think it is the responsibility of the user at that point if I specifically decide to swap out core formulas from a pinned tap. We just need some way that makes it possible to install core formulas with another configuration than the default one.
The current situation make this very difficult. On the one hand you want core to be low-mainenance with little flexility (no options), more like Debian, which is OK. But then at the same time you prevent users from swapping out formulas with a custom build, as you would do in with a custom deb/PPA. So now there is no solution anymore for users that need a non-default configuration of a formula.
my example: if a package in Homebrew/homebrew-core relies on the nss or qt functionality: the custom poppler in your tap will break that package.
Most libraries won't change their API or ABI based on the configuration. Usually only certain features of the software will be enabled/disabled. Configuring a library with or without support for some driver/lib rarely affects how the program is called or linked, usually they can be swapped out safely.
Even if it would break a reverse dependency, at that point it is my own responsibility, not yours. But I do feel strongly that if you decide to remove all flexibility in core formulas to lower mainenance, you need to relax the requirement that core take precedence over all.
I've elaborated a bit on this here: Homebrew/brew#5615 (comment). I'm open to improving the docs to make this (i.e. use a different formula name) more obvious and well-documented.
A different formula name is really not a solution, then I would also need to add all reverse dependencies of the formula to my tap with a dependency on the changed formula name. This makes the problem an order of magnitude larger.
I think almost all users in this thread simply need some way to configure a core formula with features they need.. If options are no longer supported and you present taps as the solution, you should allow core formula to be swapped out with a custom one from a tap.
I've been thinking about this for awhile now. My first interaction with the Homebrew community has been pretty negative, but I'm going to put that aside and give this another try.
@MikeMcQuaid, would you be open to the idea of an "override" tap? This would functionally be the same as a custom yum or apt repo.
The way I'm thinking this could work is that a user would specify a tap (or taps) that would be checked prior to checking the core formulae. Formulae installed from the "override" tap would satisfy the dependencies of core formulae. This would allow a user to substitute in a custom formulae, but also allow Homebrew to offload the support requests to tap maintainers. This would also help a third party tap aggregate common custom formulae into one tap.
@MikeMcQuaid, would you be open to the idea of an "override" tap? This would functionally be the same as a custom yum or apt repo.
@voldemortensen We have custom repositories (taps): https://docs.brew.sh/Taps. These can be pinned with brew tap-pin.
Formulae installed from the "override" tap would satisfy the dependencies of core formulae. This would allow a user to substitute in a custom formulae, but also allow Homebrew to offload the support requests to tap maintainers.
This will not work because these formulae will end up breaking core formulae in ways that are hard to debug (e.g. missing symbols/libraries) and we will have to be the ones fielding those support requests.
You can create your own taps with your own formulae that are installed however you like. You can't inject random formulae as dependencies for the ones we test and build and expect them to work.
I think it is the responsibility of the user at that point if I specifically decide to swap out core formulas from a pinned tap. We just need some way that makes it possible to install core formulas with another configuration than the default one.
@jeroen Regardless of whose responsibility it is: we will be the ones receiving the bug reports so we'd rather not add functionality to increase our support burden, sorry.
The current situation make this very difficult. On the one hand you want core to be low-mainenance with little flexility (no options), more like Debian, which is OK. But then at the same time you prevent users from swapping out formulas with a custom build, as you would do in with a custom deb/PPA. So now there is no solution anymore for users that need a non-default configuration of a formula.
Debian doesn't provide the same degree of user accessible support on GitHub if you have a package that breaks.
We definitely do support swapping out formulae but we do not support swapping out the dependencies of formulae because, as I mentioned in the last comment: they will break.
So now there is no solution anymore for users that need a non-default configuration of a formula.
Yes, there is: maintain your own version of the formula in a tap. As noted in Homebrew/brew#5622 the best way to do this is to use a different name so they can live side-by-side. This has been successfully working with e.g. the nginx-full formula for many years.
Most libraries won't change their API or ABI based on the configuration. Usually only certain features of the software will be enabled/disabled. Configuring a library with or without support for some driver/lib rarely affects how the program is called or linked, usually they can be swapped out safely.
"Most", "usually" and "rarely" add up to a non-trivial number of support requests at Homebrew's scale. We're not willing to support that.
Even if it would break a reverse dependency, at that point it is my own responsibility, not yours. But I do feel strongly that if you decide to remove all flexibility in core formulas to lower mainenance, you need to relax the requirement that core take precedence over all.
Again: regardless of whose responsibility it is: we field the support requests so it ends up adding extra work for us.
Core formulae remain the primary option because they are what is used by the vast majority of users. The majority of users configurations are now handled by the defaults we chose for the software in Homebrew/homebrew-core now.
A different formula name is really not a solution, then I would also need to add all reverse dependencies of the formula to my tap with a dependency on the changed formula name. This makes the problem an order of magnitude larger.
I think almost all users in this thread simply need some way to configure a core formula with features they need.. If options are no longer supported and you present taps as the solution, you should allow core formula to be swapped out with a custom one from a tap.
Again: we do allow these formulae to be swapped out but not as dependencies of existing formulae.
I don't think we can really continue discussion indefinitely on this closed issue. Where's there's specific requests for things that make life easier for tap authors without making things harder for the Homebrew maintainers: we have and will help. Where PRs are submitted that make life easier for tap authors: we will review them and may merge them.
Finally, it's worth remembering that the Homebrew maintainers do not need to do anything. In fact, open source maintainers owe you nothing.
If you want some functionality we are not willing to support: you need to be willing to do the work yourself (which may be more work than you wish given our constraints) or you need to accept what we're willing to provide you (for free, in our spare time).
You can create your own taps with your own formulae that are installed however you like. You can't inject random formulae as dependencies for the ones we test and build and expect them to work.
That's exactly how PPA's work in Ubuntu/Debian and it's fine. It's the only way to customize a formula if you don't want to support options in core formulae. Users that choose to swap out core libraries with a custom tap probably know what they are doing. It's not your responsibility to debug this.
However your current policy of hardcoding a single config for all core formulae and at the same time not providing any way to swap them out with a custom built, is obviously leading to frustration by many users for which the default configuration is inappropriate.
That's exactly how PPA's work in Ubuntu/Debian and it's fine.
It's not exactly how they work because they both ship binaries and Homebrew/homebrew-core ship binaries but taps/options ship source builds.
Even if it were literally identical: It's perhaps fine to them/you but t's not fine to us. It will create bugs that are not immediately obvious, it will create user confusion and it will create extra work for us (even if we could instantly detect these issues and close them).
I'm relocking this thread in favour of our Discourse forum for more general discussion and specific issues (e.g. emacs cask and formulae conflict
https://github.com/Homebrew/homebrew-core/issues/363109 or specific regressions with new options
#36342) or PRs that add functionality to Homebrew/brew or make change the default build in Homebrew/homebrew-core when relevant.
I’ve reopened the Discourse thread Maintaining a tap to provide options for a formula to discuss possible PRs that help tap authors do their work with less friction.
@bumfo Feel free to follow up there if you wish.