Alchemist Resurrection
kentbull opened this issue · 21 comments
Time to resurrect this project. Whoever wants to help then respond to this issue and let's get some plans ready. At the very least we need to do a detailed comparison of Alchemist to Elixir Mode (https://github.com/elixir-editors/emacs-elixir) and patch into Alchemist the things missing. We also need to inventory all open issues, evaluate, and prioritize them.
Let's look at the forks and see if any of them are much further than this one.
Here we go! If Tonini decides to return back from his vacation then we'll welcome the grandmaster himself. Until then let's get this mischief managed :)
The ElixirLS (https://github.com/elixir-lsp/elixir-ls) project has gotten quite some traction because Alchemist was kind of abandoned. ElixirLS works again an LSP frontend that is either lsp-mode or eglot. Not sure if you know.
So overall I'm not sure what the gain is for Alchemist compared to ElixirLS today. There are probably some things that Alchemist still does more or better. Not sure.
@mdbergmann Yes, I've used Elixir LS. There is one main bug I'm hitting with elixir-editors/emacs-elixir#415 that is part of my motivation for bringing Alchemist back to life. I also generally find Alchemist more useful and I like the name. I am in the process of doing a comparison. Maybe it doesn't make sense to bring Alchemist back to life. It seems like at the very least it would be a good way to test my learning of ELisp and help me to learn about Emacs, Elixir, and language support in editors.
Hi!
@kentbull -- development on alchemist.el has stopped. As a community, we are focusing on LSP with https://github.com/elixir-lsp (which provides one server implementation and simple plugins for each editor).
I know that some folks are still using alchemist, and the name is certainly cooler.
This wasn't an easy decision to make, because alchemist has been such a powerful tool for so long, and quite the stepping stone for much of the work (in fact, some of the tools use or used code from alchemist).
All my efforts to bring alchemist.el back to life eventually culminated into beginning the work on elixir-ls instead and focusing on making that successful -- #341
Re: elixir-editors/emacs-elixir#415 I regret merging the formatter into elixir-mode and I think we should remove it entirely and let other solutions (including a separate package dedicated to that, if we so choose) handle formatting through mix.
With elixir-ls, we are at a point where all the plugin maintainers are together under the LSP umbrella (or at least all the maintainers I know), and we also have support from the creator of elixir_sense, the project analysis tool that underpins all the work for the server.
As the Elixir community, we're better off focusing our efforts there.
For emacs-specific focus, there's a lot of tiny pieces of work that could / should be broken off of alchemist and turned into their own projects, such as something like projectile-phoenix, to do discovery / quick-jump to controllers, views, etc, or additional support for the unit-testing package, or ... Maybe a standalone formatter package -- you could probably fork emacs-elixir and take the formatter code and go from there, if you wanted to work on that specifically.
As far as the integrated development environment that alchemist provides, though -- those efforts should move over to the LSP server and clients.
Thanks for the heads up. I'll dig deeper into elixir-lsp. Whatever value there was in Alchemist I'd like to merge into the current community's efforts. Thats' as much a resurrection as this needs. I like some of the features in Alchemist that I use regularly and I'd love to add them to elixir-lsp.
I appreciate Alchemist's feature: Documentation lookup.
As a community, we are...
As the Elixir community, we're better off...
I don't want to sound harsh, but as the Emacs community, we may actually want a good language tooling, that goes beside the scope of LSP. Overall the point "as the Elixir community" sounds way too opinionated and selfish IMO.
Language server indeed provides good features, such as completion, hovering, jumping to definition, finding references, symbol search, diagnostics, and debugging, but thing like Alchemist can be much more than that, and leverage Emacs' specific things unavailable in other editor, without relying on LSP at all, and more importantly complementing what LSP can offer.
I don't disregard the work done on language server for Elixir, just want to say that Emacs can actually provide more tooling for the language than just an LSP support.
I don't disregard the work done on language server for Elixir, just want to say that Emacs can actually provide more tooling for the language than just an LSP support.
+1
LSP really is just the GCD. There is much more in to it that that for a real IDE.
@andreyorst @mdbergmann I don't know how else to say this; it's going to be rude, and I'm sorry about it.
Have you read the entire message you're criticizing? Specifically:
For emacs-specific focus, there's a lot of tiny pieces of work that could / should be broken off of alchemist and turned into their own projects, such as something like projectile-phoenix, to do discovery / quick-jump to controllers, views, etc, or additional support for the unit-testing package, or ... Maybe a standalone formatter package -- you could probably fork emacs-elixir and take the formatter code and go from there, if you wanted to work on that specifically.
If you think there should be one big package like alchemist that plugs in all of those, then that's fine too -- the actual implementation is irrelevant. I'm specifically saying that there's two orthogonal sets of concerns, and LSP solves one of those sets of concerns really well. I am not saying the other set of concerns does not exist.
Have you read the entire message you're criticizing? Specifically:
For emacs-specific focus, there's a lot of tiny pieces of work that could / should be broken off of alchemist and turned into their own projects
Yes, I disagree. This particular point I've addressed in Elixir-mode issue tracker. Here I was really pissed off by "as a community" thing, sorry for that. (I don't even write in Elixir)
I do think, that Emacs mode for Elixir should be as independent from LSP as possible, and LSP should be just another way of providing intelligent tooling for the language. This is what Alchemist was as far as I'm concerned. First off, this good because it is encourages competition. If Alchemist is better than Elixir-LS, then maybe more people will start using Emacs. If Elixir-LS is better than Alchemist, maybe we can enchance Alchemist to make it a equal or better tool. LSP is very popular this days, but there are still other ways of making smart interactive development environments. For example language plugins for Intellij IDEA usually are built on top of how Intellij can analyze the language and not on LSP. And some languages have much better support in Intellij specifically because of that, e.g. Intellij Rust vs RLS. And Rust community focused on different reimplementation of language server Rust Analyser, that will have feature parity with Intellij Rust plugin.
So to rephrase this in a more short way: Alchemist indeed should be resurrected, and community effort of building LSP independent solutions should not be discouraged.
So let me get this clear:
- you are not an Elixir user
- you came to an Elixir project to complain about something which isn't related to the project
- the piece you actually want to talk about, you didn't mention in your post, which means if I hadn't brought it up, you'd probably have left your post as a rant on a topic which isn't relevant to the actual conversation
Did I capture this correctly?
Okay, well, either way, let's move on to possibly more productive things.
that Emacs mode for Elixir should be as independent from LSP as possible, and LSP should be just another way of providing intelligent tooling for the language.
Good news, this isn't the emacs-mode project, which means we don't have to fight over this in this conversation.
this good because it is encourages competition. If Alchemist is better than Elixir-LS, then maybe more people will start using Emacs.
I have literally just finished saying that the language server and the emacs tooling are orthogonal, so either you did not understand or you disagree. Two concerns being orthogonal means they're unrelated and can be solved independently. In this case, what I mean is, we can do elixir-ls AND we can do emacs tooling as two completely separate efforts
Alchemist indeed should be resurrected, and community effort of building LSP independent solutions should not be discouraged.
Great. this statement brings us back to the topic at hand.
Back when this issue was written, six months or so ago (and from a human perspective, at least in the US, a lifetime ago), elixir-ls hadn't yet reached the level of grounding that it now has. Here's some of the things we've achieved:
- contact with the erlang-ls team, to find ways to share efforts and common infrastructure
- gathering of the emacs, vim, vscode lsp clients in the same github org
- collaboration with the author of elixir_sense
- fairly regular releases and improvements
Which means we're doing alright from that end. now, I have the energy to look at this effort.
First, @andreyorst , I have zero understanding of why you want "alchemist" over "a set of smaller tools", so I'd like you to explain the trade-offs you envision in this choice and why you prefer "one big tool". I could guess, but that might be insulting and wrong.
Second, since I had originally tried to resurrect alchemist.el to make improvements to it, and then explored the code, I like to think I have a sense of the current complexity of the codebase. In order to make alchemist work, Tonini had to manage multiple layers:
- interaction with the system (find mix, find elixir)
- interaction with mix (run mix commands, get results)
- some code that became elixir_sense ( https://github.com/tonini/alchemist.el/tree/master/alchemist-server ) and has been unmaintained for three years now, while elixir_sense is maintained (see list of elixir-ls successes)
- interaction with emacs (windows, inferior modes, etc.)
A very large amount of alchemist's functionality rests on the alchemist server, which means a lot of the important updates to alchemist go through updating this. This led me down the path of trying to replace it with elixir_sense (at the time, a few years ago, I had a hard time reaching the maintainer, and had decided to fork it for this purpose), and then to replace it with elixir-ls (same, forked it for this purpose). As I tried to find the correct seam in the code to do the replacement, I eventually realized it would be too much work, and decided to split entirely from alchemist to focus only on elixir-ls.
Since alchemist has gone unmaintained, some key efforts were born, such as:
- the mix package
- the exunit package
These are fairly fundamental efforts, and I'm glad to see them exist. Smaller efforts are more maintainable, by more people.
All this being said, if @kentbull wants to do this work, then I hope he does, and I hope others help him!
As I mentioned above, I regret merging in the code formatter in elixir-mode and I would rather a package other than elixir-mode handle formatting that uses an external package. I did not imagine there'd be so many headaches with that formatter, and so few people willing/able to help solve them.
Did I capture this correctly?
Not quite. I do plan to use Elixir in the future, and after some small experiments I was unhappy with current state of LSP, and overall tooling, so I came to Emacs project related at providing good support for Elixir, which is still among the top links in the Google. And it seems to me that I am not the only one here.
Good news, this isn't the emacs-mode project
I fail to see how it is not a Emacs-mode when the project even has the .el in the name, written in Emacs lisp and works only in Emacs. Perhaps you've meant not Elixir-mode?
I have literally just finished saying that the language server and the emacs tooling are orthogonal
orthogonal things do not compete, so I'm not sure how it is relevant to what I'm saying. What I've meant is that for example Emacs' semantic.el could be used instead of LSP. Not suggesting that it should really be done via semantic though.
The rest things you wrote I do agree. Mix.el and exunit.el is good thing to have. But looking at a pretty successful projects like SLIME or CIDER, I still think that a more complete solution is the way to go. It does need a more in depth architecture thought through indeed.
so there hasn't been a commit since 2018...seems like there is nobody to review PRs? is there a fork?
Is the alternative Elixir Ls?
Yeah, nobody to review PRs. I'm up for it if someone wants to do it with me. I've learned a fair bit about Emacs and Elixir since I first started. Yet I also didn't want to bring this project up to speed since emacs-elixir is out now. In order to justify this project there has to be value add in comparison with emacs-elixir.
I share the same mindset of @Trevoke and prefer small packages for specific purposes. This seams to be the mindset of Emacs community as well. Big packages like magit breaking the implementation in many small packages. Or specific packages being born like vertico as alternative to ivy and helm.
I think this issue got out of hand, I've been using this package in Emacs for a couple of weeks now, and even tho the features are not that great (compared to the LSP server). It work as intended, and has the potential to be quite better, I think it just need a little bit of love.
If nobody wants to maintain the package, I can try to merge the request and add some things in a fork, I'll add a comment in the Melpa issue to make people aware that this package is dormant.
I think Alchemist can be a nice alternative to the LSP version, maybe with less features, but more Emacs integrated (not LSP dependant), I'm not a big fan of LSP, (even tho I'm the maintainer of lsp-sonarlint) and also I'm a big fan of having choice in therm of development tools.
For now, I'm merging some pull requests here, and I think in the near future I'll try to change the back-end from alchemist-server to elixir_sense (as the vim extension did some years ago)
It's not my goal to start a fight between LSP vs Emacs, we can all benefit from a solid back-end and common tools, I think the Clojure community did it pretty well, they develop nrepl so CIDER(Emacs version) and CALVA(VSCode version) can use it.
To wrap it up, I don't know if I have the time nor the knowledge to write everything that this project need, but I'll try to make some changes so at least the project is stable and ready to use. (even tho now is usable, I think it does have some major bugs/usability problems)
A quick update, I decied to modify the main alchemist and make breaking changes (I changed the name of the project, not to confuse people).
Now that eglot is in core Emacs, the main idea is to remove from the package the funcionalitites that the lsp server can do (basically most of the things that involves the alchemist-server). Focus more the package functionality to the REPL, documentation and overall generic interaction with Emacs.
I still wanto to use the REPL to have some funcionality that for now is bound to the server (like macroexpand).
Like I said, I merge most of the merge requests and I'm going to take the oportunity to make the code a little bit more cl-lib oriented (sorry dash) and adding some nice functionality (maybe with tree-sitter now that this is also in core Emacs).
My fork is called Apprentice and it can be found here https://github.com/Sasanidas/Apprentice
This is exciting! Nice work @Sasanidas. I ended up moving into the Rust and Clojure(Script) spaces though I still have a special place in my heart for Elixir. I'm glad you're taking this on.