Request: update to use hsdev?
Closed this issue · 18 comments
ghc-mod is quite outdated by now, and hardly even installs with the current tools.
It would be great if this plugin were updated to use hsdev instead (or maybe intero).
Would you consider doing it?
I'm definitely fine with adding new backends. We have somewhat of an infrastructure for that sort of thing in HaskForce, so it's not too bad adding them. I don't think I have the bandwidth for that at the moment, but would be happy to outline the details of implementing something like this if someone is interested in contributing.
One important question is, what features does hsdev have that we expect to leverage? We should enumerate these first so it's clear whether the existing backend infrastructure is sufficient enough or if we need to add additional abstractions.
Also note that ghc-mod is still leveraged often by other tools from what I can tell. In particular, it serves as the core of haskell-ide-engine, which I'm also very interested in getting integrated as a backend to HaskForce (see #223).
What features does hsdev have...?
It has the most important feature: it builds under the current cabal/stack/ghc, while ghc-mod does not.
So it's the issue of replacing something that's dead (at least on MacOS) with something alive.
Depending on the complexity of this Enterprise, I might be able to help - but being a beginner in Haskell, in not sure how successful it would be.
I use ghc-mod on Mac OSX nearly every day. I think it's mostly a matter of automating the building, configuration, and integration of external tools via the IDE, whether they be ghc-mod, intero, etc.
Most likely you don't have Macports installed, and/or no port that depends on Macports libiconv.dylib. I can't provide a reference to a set of issues related to this (searching and pasting URLs from a cell phone is less than convenient - you can take my word on this, or check for yourself) - the main Haskell-related problem is that many packages (especially the big ones like intero, which rely on many dependencies) fail to link. Collision between two shared libraries with the same name but different exposed function names. Cabal seems more respectful of the global config parameters, and can build/install almost everything (still, there are some exceptions, like ghc-mod). Stack fails a good portion of packages.
So, is a matter of what tools to keep, and which ones to drop. The majority of my development is not in Haskell, so Macports stays. And on that setup ghc-mod fails building no matter what (cabal or stack).
This is why I'm looking for a solution that could interoperate with my setup. That would require a plug-in that does not friend on ghc-mod mod, as I cannot build it locally.
P.S. In case it matters - this libiconv issue is what prevents me from Judy using intellij-haskell plugin straight out-of-box: stack fails to build things.
P.P.S. Could you share how you build ghc-mod on your setup? Perhaps I can replicate it...
Here's an example. Despairing of installing directly, I cloned ghc-mod from GitHub:
$ cd ghc-mod
$ cabal new-install
Resolving dependencies...
Up to date
Warning: You asked to install executables, but there are no executables in
targets: . Perhaps you want to use --lib to install libraries instead.
$ cabal build
Warning: The build command is a part of the legacy v1 style of cabal usage.
Please switch to using either the new project style and the new-build command
or the legacy v1-build alias as new-style projects will become the default in
the next version of cabal-install. Please file a bug if you cannot replicate a
working v1- use case with the new-style commands.
For more information, see: https://wiki.haskell.org/Cabal/NewBuild
Warning: ghc-mod.cabal:98:10: Tabs used as indentation at 98:10, 160:10
Resolving dependencies...
Warning: solver failed to find a solution:
Could not resolve dependencies:
[__0] trying: ghc-mod-5.9.0.0 (user goal)
[__1] next goal: base (dependency of ghc-mod)
[__1] rejecting: base-4.12.0.0/installed-4.1... (conflict: ghc-mod =>
base<4.11 && >=4.6.0.1)
[__1] fail (backjumping, conflict set: base, ghc-mod)
After searching the rest of the dependency tree exhaustively, these were the
goals I've had most trouble fulfilling: ghc-mod, base
Trying configure anyway.
Warning: ghc-mod.cabal:98:10: Tabs used as indentation at 98:10, 160:10
[1 of 1] Compiling Main ( dist/setup/setup.hs, dist/setup/Main.o )
dist/setup/setup.hs:18:5: error:
• Couldn't match expected type ‘UserHooks’
with actual type ‘Cabal-2.4.0.1:Distribution.Simple.UserHooks.UserHooks’
NB: ‘Cabal-2.4.0.1:Distribution.Simple.UserHooks.UserHooks’
is defined in ‘Distribution.Simple.UserHooks’
in package ‘Cabal-2.4.0.1’
‘UserHooks’
is defined in ‘Distribution.Simple.UserHooks’
in package ‘Cabal-2.4.1.0’
• In the second argument of ‘($)’, namely
‘addDoctestsUserHook "doctest"
$ simpleUserHooks {hookedPrograms = [simpleProgram "shelltest"]}’
In the expression:
defaultMainWithHooks
$ addDoctestsUserHook "doctest"
$ simpleUserHooks {hookedPrograms = [simpleProgram "shelltest"]}
In an equation for ‘main’:
main
= defaultMainWithHooks
$ addDoctestsUserHook "doctest"
$ simpleUserHooks {hookedPrograms = [simpleProgram "shelltest"]}
|
18 | addDoctestsUserHook "doctest" $
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^...
dist/setup/setup.hs:19:5: error:
• Couldn't match expected type ‘Cabal-2.4.0.1:Distribution.Simple.UserHooks.UserHooks’
with actual type ‘UserHooks’
NB: ‘UserHooks’
is defined in ‘Distribution.Simple.UserHooks’
in package ‘Cabal-2.4.1.0’
‘Cabal-2.4.0.1:Distribution.Simple.UserHooks.UserHooks’
is defined in ‘Distribution.Simple.UserHooks’
in package ‘Cabal-2.4.0.1’
• In the second argument of ‘($)’, namely
‘simpleUserHooks {hookedPrograms = [simpleProgram "shelltest"]}’
In the second argument of ‘($)’, namely
‘addDoctestsUserHook "doctest"
$ simpleUserHooks {hookedPrograms = [simpleProgram "shelltest"]}’
In the expression:
defaultMainWithHooks
$ addDoctestsUserHook "doctest"
$ simpleUserHooks {hookedPrograms = [simpleProgram "shelltest"]}
|
19 | simpleUserHooks {
| ^^^^^^^^^^^^^^^^^...
$ stack install
Error parsing targets: The specified targets matched no packages.
Perhaps you need to run 'stack init'?
$
Update
And here are the commands that hsdev supports:
$ hsdev --help
Usage: hsdev (COMMAND | ([--port number] | [--unix name]) [--pretty] [--stdin]
[--timeout msec] [--silent] [--no-file] COMMAND)
hsdev tool
Available options:
-?,--help show help
--port number connection port
--unix name unix connection port
--pretty pretty json output
--stdin pass data to stdin
--timeout msec query timeout
--silent supress notifications
--no-file don't use mmap files
Available commands:
version hsdev version
start start remote server
run run server
stop stop remote server
connect connect to send commands directly
ping ping server
listen listen server log
set-log set log level
scan scan sources
set-file-contents set edited file contents, which will be used instead
of contents in file until it updated
docs scan docs
infer infer types
remove remove modules info
remove-all remove all data
packages list packages
projects list projects
sandboxes list sandboxes
symbol get symbol info
module get module info
project get project info
sandbox get sandbox info
lookup lookup for symbol
whois get info for symbol
whoat get info for symbol under cursor
scope get declarations accessible from module or within a
project
usages find usages of symbol within project/module
complete show completions for input
hayoo find declarations online via Hayoo
cabal cabal commands
unresolveds list unresolved symbols in source file
lint lint source files or file contents
check check source files or file contents
check-lint check and lint source files or file contents
types get types for file expressions
autofixes get autofixes by output messages
refactor apply some refactors and get rest updated
rename get rename refactors
ghc ghc commands
langs ghc language options
flags ghc flags
link link to server
stop-ghc stop ghc sessions
exit exit
$
I have some steps documented for getting ghc-mod working here - http://caryrobbins.com/dev/installing-ghc-mod/
I'm open to PRs for hsdev support, we'd just need to have a plan for exactly which commands we want to leverage, what those interfaces are, and whether or not we have the infrastructure currently in place.
I have some steps documented for getting ghc-mod working here...
Thanks! Very good description. Unfortunately, with a new installation it still seems to fail:
$ git clone https://github.com/alanz/ghc-mod
Cloning into 'ghc-mod'...
remote: Enumerating objects: 86, done.
remote: Counting objects: 100% (86/86), done.
remote: Compressing objects: 100% (54/54), done.
remote: Total 15294 (delta 53), reused 55 (delta 32), pack-reused 15208
Receiving objects: 100% (15294/15294), 3.22 MiB | 8.07 MiB/s, done.
Resolving deltas: 100% (8241/8241), done.
$ cd ghc-mod
$ git checkout ghc-8.4-hie
Branch 'ghc-8.4-hie' set up to track remote branch 'ghc-8.4-hie' from 'origin'.
Switched to a new branch 'ghc-8.4-hie'
$ cabal update
Warning: The update command is a part of the legacy v1 style of cabal usage.
Please switch to using either the new project style and the new-update command
or the legacy v1-update alias as new-style projects will become the default in
the next version of cabal-install. Please file a bug if you cannot replicate a
working v1- use case with the new-style commands.
For more information, see: https://wiki.haskell.org/Cabal/NewBuild
Downloading the latest package list from hackage.haskell.org
To revert to previous state run:
cabal update --index-state='2019-03-28T00:21:45Z'
$ cabal sandbox init
Warning: The sandbox command is a part of the legacy v1 style of cabal usage.
Please switch to using either the new project style and the new-sandbox
command or the legacy v1-sandbox alias as new-style projects will become the
default in the next version of cabal-install. Please file a bug if you cannot
replicate a working v1- use case with the new-style commands.
For more information, see: https://wiki.haskell.org/Cabal/NewBuild
Writing a default package environment file to
/Users/ur20980/src/ghc-mod/cabal.sandbox.config
Creating a new sandbox at /Users/ur20980/src/ghc-mod/.cabal-sandbox
$ vi cabal.sandbox.config
$ cabal sandbox add-source core
Warning: The sandbox command is a part of the legacy v1 style of cabal usage.
Please switch to using either the new project style and the new-sandbox
command or the legacy v1-sandbox alias as new-style projects will become the
default in the next version of cabal-install. Please file a bug if you cannot
replicate a working v1- use case with the new-style commands.
For more information, see: https://wiki.haskell.org/Cabal/NewBuild
$ type happy
happy is /Users/ur20980/Library/Haskell/bin/happy
$ cabal install happy
Warning: The install command is a part of the legacy v1 style of cabal usage.
Please switch to using either the new project style and the new-install
command or the legacy v1-install alias as new-style projects will become the
default in the next version of cabal-install. Please file a bug if you cannot
replicate a working v1- use case with the new-style commands.
For more information, see: https://wiki.haskell.org/Cabal/NewBuild
Warning: the following files would be used as linker inputs, but linking is not being done: /usr/lib/libiconv.dylib /usr/lib/libiconv.dylib
Resolving dependencies...
Notice: installing into a sandbox located at
/Users/ur20980/src/ghc-mod/.cabal-sandbox
Starting happy-1.19.9
Building happy-1.19.9
Completed happy-1.19.9
Updating documentation index
/Users/ur20980/src/ghc-mod/.cabal-sandbox/share/doc/x86_64-osx-ghc-8.6.3/index.html
$ cabal install
Warning: The install command is a part of the legacy v1 style of cabal usage.
Please switch to using either the new project style and the new-install
command or the legacy v1-install alias as new-style projects will become the
default in the next version of cabal-install. Please file a bug if you cannot
replicate a working v1- use case with the new-style commands.
For more information, see: https://wiki.haskell.org/Cabal/NewBuild
Warning: the following files would be used as linker inputs, but linking is not being done: /usr/lib/libiconv.dylib /usr/lib/libiconv.dylib
Resolving dependencies...
cabal: Could not resolve dependencies:
[__0] trying: ghc-mod-5.9.0.0 (user goal)
[__1] next goal: base (dependency of ghc-mod)
[__1] rejecting: base-4.12.0.0/installed-4.1... (conflict: ghc-mod =>
base<4.12 && >=4.6.0.1)
[__1] rejecting: base-4.12.0.0, base-4.11.1.0, base-4.11.0.0, base-4.10.1.0,
base-4.10.0.0, base-4.9.1.0, base-4.9.0.0, base-4.8.2.0, base-4.8.1.0,
base-4.8.0.0, base-4.7.0.2, base-4.7.0.1, base-4.7.0.0, base-4.6.0.1,
base-4.6.0.0, base-4.5.1.0, base-4.5.0.0, base-4.4.1.0, base-4.4.0.0,
base-4.3.1.0, base-4.3.0.0, base-4.2.0.2, base-4.2.0.1, base-4.2.0.0,
base-4.1.0.0, base-4.0.0.0, base-3.0.3.2, base-3.0.3.1 (constraint from
non-upgradeable package requires installed instance)
[__1] fail (backjumping, conflict set: base, ghc-mod)
After searching the rest of the dependency tree exhaustively, these were the
goals I've had most trouble fulfilling: ghc-mod, base
Note: when using a sandbox, all packages are required to have consistent
dependencies. Try reinstalling/unregistering the offending packages or
recreating the sandbox.
$
I'm open to PRs for hsdev support,
Thanks!
...we'd just need to have a plan for exactly which commands we want to leverage, what those interfaces are, and whether or not we have the infrastructure currently in place
A bigger question would probably be - which commands we need. Above is a list of what hsdev seems to provide - what does this plugin want from it, or from ghc-mod?
Regarding your ghc-mod issue, I wonder if the problem might be related to the version of cabal-install you are running (in the post I linked I'm using cabal-install version 2.2.0.0). I'm not saying to use that version (not sure how that will affect your system), but that may be part of the problem. Automating the installation of tooling I think would help this, regardless of the backend used.
Regarding your ghc-mod issue, I wonder if the problem might be related to the version of cabal-install you are running
I've no idea - in Haskell I'm a true beginner. And its ecosystem with no stability whatsoever (you want something to run - throw together every dependency for that version exactly) drives me mad. But it is what it is.
Automating the installation of tooling I think would help this
I'm all for automating the installation - how would you suggest I proceed with it? Given my current setup (as I can hardly go back to cabal-2.2.0.0, for example)?
Update
Cary, it looks like hhp from Kazu got installed without a hitch:
$ cabal install hhp
Warning: The install command is a part of the legacy v1 style of cabal usage.
Please switch to using either the new project style and the new-install
command or the legacy v1-install alias as new-style projects will become the
default in the next version of cabal-install. Please file a bug if you cannot
replicate a working v1- use case with the new-style commands.
For more information, see: https://wiki.haskell.org/Cabal/NewBuild
Warning: the following files would be used as linker inputs, but linking is not being done: /usr/lib/libiconv.dylib /usr/lib/libiconv.dylib
Resolving dependencies...
Downloading io-choice-0.0.7
Downloaded io-choice-0.0.7
Starting io-choice-0.0.7
Building io-choice-0.0.7
Completed io-choice-0.0.7
Downloading hhp-0.0.0
Downloaded hhp-0.0.0
Starting hhp-0.0.0
Building hhp-0.0.0
Completed hhp-0.0.0
Updating documentation index
/Users/ur20980/Library/Haskell/share/doc/x86_64-osx-ghc-8.6.3/index.html
$ type -all hhpi
hhpi is /Users/ur20980/.cabal/bin/hhpi
hhpi is /Users/ur20980/Library/Haskell/bin/hhpi
$
Can it be used by your plugin instead of ghc-mod?
Update 2a
Preprocessing library for cabal-helper-0.8.1.2..
Building library for cabal-helper-0.8.1.2..
creating dist/build
/Users/ur20980/.ghcup/bin/ghc --make -fbuilding-cabal-package -O -static -dynamic-too -dynosuf dyn_o -dynhisuf dyn_hi -outputdir dist/build -odir dist/build -hidir dist/build -stubdir dist/build -i -idist/build -ilib -isrc -idist/build/autogen -idist/build/global-autogen -Idist/build/autogen -Idist/build/global-autogen -Idist/build -I/opt/local/include -I/usr/local/include -optP-include -optPdist/build/autogen/cabal_macros.h -this-unit-id cbl-hlpr-0.8.1.2-6e307d73 -hide-all-packages -Wmissing-home-modules -no-user-package-db -package-db /Users/ur20980/.cabal/store/ghc-8.6.3/package.db -package-db dist/package.conf.inplace -package-id Cabal-2.4.0.1 -package-id base-4.12.0.0 -package-id cbl-pln-0.5.0.0-e940cdc9 -package-id containers-0.6.0.1 -package-id directory-1.3.3.0 -package-id filepath-1.4.2.1 -package-id mtl-2.2.2 -package-id process-1.6.3.0 -package-id smgrpds-5.3.2-5c9fcf8e -package-id transformers-0.5.5.0 -package-id unix-2.7.2.2 -package-id nx-cmpt-0.5.1-641f3e2d -XHaskell2010 -XNondecreasingIndentation Distribution.Helper CabalHelper.Shared.InterfaceTypes CabalHelper.Shared.Sandbox Paths_cabal_helper -Wall -hide-all-packages -pgmc clang -pgma clang /usr/lib/libiconv.dylib -pgmc clang -pgma clang /usr/lib/libiconv.dylib -pgmc clang -pgma clang /usr/lib/libiconv.dylib -pgmc clang -pgma clang /usr/lib/libiconv.dylib
[1 of 4] Compiling CabalHelper.Shared.InterfaceTypes ( src/CabalHelper/Shared/InterfaceTypes.hs, dist/build/CabalHelper/Shared/InterfaceTypes.o )
[2 of 4] Compiling CabalHelper.Shared.Sandbox ( src/CabalHelper/Shared/Sandbox.hs, dist/build/CabalHelper/Shared/Sandbox.o )
[3 of 4] Compiling Paths_cabal_helper ( dist/build/autogen/Paths_cabal_helper.hs, dist/build/Paths_cabal_helper.o )
[4 of 4] Compiling Distribution.Helper ( lib/Distribution/Helper.hs, dist/build/Distribution/Helper.o )
lib/Distribution/Helper.hs:383:3: error:
• Could not deduce (Control.Monad.Fail.MonadFail m)
arising from a do statement
with the failable pattern ‘[Just (ChResponseVersion pkgName
pkgVer)]’
from the context: MonadQuery m
bound by the type signature for:
getPackageId :: forall (m :: * -> *).
MonadQuery m =>
m (String, Version)
at lib/Distribution/Helper.hs:381:1-51
Possible fix:
add (Control.Monad.Fail.MonadFail m) to the context of
the type signature for:
getPackageId :: forall (m :: * -> *).
MonadQuery m =>
m (String, Version)
• In a stmt of a 'do' block:
[Just (ChResponseVersion pkgName pkgVer)] <- readHelper
["package-id"]
In the expression:
do [Just (ChResponseVersion pkgName pkgVer)] <- readHelper
["package-id"]
return (pkgName, pkgVer)
In the second argument of ‘(>>=)’, namely
‘\ QueryEnv {..}
-> do [Just (ChResponseVersion pkgName pkgVer)] <- readHelper [...]
return (pkgName, pkgVer)’
|
383 | [ Just (ChResponseVersion pkgName pkgVer) ] <- readHelper [ "package-id" ]
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Warning: Some package(s) failed to build. Try rerunning with -j1 if you can't
see the error.
@carymrobbins one problem that prevents me from contributing is that I don't understand how this plugin functions, and where the "interaction points" with ghc-mod are. Plus, I'm quite decent with Java, but never touched Scala.
I understand you're busy - but perhaps you can point me at those places in the code where it
- instantiates/invokes
ghc-mod; - passes requests to it and gets responses;
- shuts it down;
- other interactions that I don't know enough to ask about.
Armed with that knowledge, I might be able to better contribute to moving this plugin from the currently-dead ghc-mod to something else, like hhp or hsdev.
You keep saying that ghc-mod is dead, but it is at the core of the haskell-ide-engine, which is actively being developed.
PR #368 was recently merged which added auto-import support. The GhcModi class manages most of the spawning, killing, and interacting with ghc-mod. In that PR you'll see that we have a SymbolImportProvider and SymbolImportProviderFactory for delegating finding unknown symbols to ghc-mod, and then an AddToImports intention for actually inserting the import into the AST. There's obviously a lot more to it but it becomes more intuitive as you begin working on the project.
You keep saying that
ghc-modis dead, but it is at the core of thehaskell-ide-engine, which is actively being developed.
That's what the more-knowledgable-than-me people on the #ghc channel said, quoting an updated README page that stated: "using ghc-mod as a user facing tool for IDE/Editor integration is no longer supported or maintained".
The fact that it's practically impossible to even build ghc-mod with cabal-2.4.0.0 and ghc-8.6.3 speaks for itself. haskell-ide-engine is indeed being developed - but it's not integrated with this plugin, and I don't know yet if it builds OK in my environment. As I said, stack exhibits far more problems for me than cabal, so far.
Thanks for the references - I'll look and work through them.
Update: failed attempt to build HIE: haskell/haskell-ide-engine#1154
Next sentence 😉
Currently haskell-ide-engine still uses ghc-mod as a library for GHC session setup, so this part will still be maintained.
Honestly, I think putting the effort toward haskell-ide-engine is probably more worthwhile. Maybe hsdev is better, but the goal of haskell-ide-engine is a little more clear (and theoretically supports the notion of using something like hsdev as an optional backend). However, feel free to evaluate them. I'll probably look more into hsdev now anyway.
However, feel free to evaluate them
I'm not sure I'm qualified to give a rounded evaluation - I'm looking at those only from the point of "will it work with a plugin that works with Intellij IDEA". For that "it" would have to at least build on my machines (every one of which has Macports installed). And from that point of view - hsdev works, hhp works (both build OK via Cabal, and are invokable). ghc-mod doesn't, and hie doesn't - they don't even build.
As you saw in my previous post, HIE failed to build: haskell/haskell-ide-engine#1154. I suspect it's the problem with stack being unable/unwilling to pass linker parameters to dependencies down the chain. So I'm not crazy about switching to it, unless that failure can be rectified (and I truly hope it would be, but I'm not holding my breath).
My main interest in this plugin comes from its configurability. I hope it can be divorced from stack. I already created "External Tools" in IDEA, and was able using this plugin to build and run a toy project (but of course, without the benefits that Haskell front-end like ghc-mod would provide).
For what it's worth, I'm trying out hsdev a little to see how it goes. Unfortunately, I've run into this issue upon an initial scan of a project.
It is extraordinarily convenient that it is directly buildable with stack though, so that gives some hope (other IDE tools seem to have complicated, painful builds).
Update: I've only just played with hsdev locally preliminarily, but it seems really easy to install and use, and it's fast, even on a large project. My time is going to be pretty limited this week, but I may be able to (soonish) have something going with hsdev in HaskForce.
Good to hear.
As you can see from the slew of commits above, I've put a bunch of work into getting hsdev going in HaskForce. Unfortunately, there are some usability problems with it currently and it doesn't quite seem ready to integrate just yet. I've opened some issues on the hsdev project to address them. I'm leaving the work out there on the 381-hsdev branch but likely won't be merging it any time soon.