Language Server does not build for WebAssembly anymore

Question

Language Server does not build for WebAssembly anymore

turbolent opened this issue 2 years ago · 10 comments

Problem

Attempting to compile the Language Server for WebAssembly fails, though it used to work:

Steps to Reproduce

$ cd languageserver
$ make wasm
GOARCH=wasm GOOS=js go build -o languageserver.wasm  ./cmd/languageserver
package github.com/onflow/cadence-tools/languageserver/cmd/languageserver
	imports github.com/onflow/cadence-tools/languageserver/server
	imports github.com/onflow/cadence-tools/lint
	imports github.com/onflow/flow-cli/pkg/flowkit/gateway
	imports github.com/onflow/flow-emulator
	imports github.com/onflow/flow-emulator/convert
	imports github.com/onflow/flow-emulator/convert/sdk
	imports github.com/onflow/flow-go/access
	imports github.com/onflow/flow-go/consensus/hotstuff
	imports github.com/onflow/flow-go/module
	imports github.com/onflow/flow-go/state/protocol
	imports github.com/onflow/flow-go/storage
	imports github.com/dgraph-io/badger/v2
	imports golang.org/x/sys/unix: build constraints exclude all Go files in /Users/bastian/go/pkg/mod/golang.org/x/sys@v0.2.0/unix
make: *** [wasm] Error 1

Acceptance Criteria

Compilation to WebAssembly should succeed like it used to.

Answer 1 · 2023-02-10T21:18:40.000Z

Investigation

In the cadence repo, tag languageserver/v0.26.2 (last tag there) works 👍
In cadence-tools:
- Tag languageserver/v0.2.0 (first tag there) does not work 👎
- Earlier commit 885c7c3727f17f7c5b50ffb1168ef200de242eb8 ("Update LS to Cadence v0.27.0", PR #27) works 👍
- git bisecting reveals f46efb5 still works 👍
- git diff f46efb5..4e04255 shows flowkit is updated from dfd5770b1cd7 to 8d43a4be0028
- Looking at what changed in flowkit: onflow/flow-cli@dfd5770...8d43a4b
  - flowkit adds the test framework (cadence-tools/test): onflow/flow-cli@dfd5770b1cd7...8d43a4be0028#diff-1b4735b99c

Problem

The language server depends on flowkit
flowkit started to depend on the test framework
The test framework depends on the emulator
The Emulator does not support compilation to WebAssembly

Potential solutions

Get the Emulator to compile to WebAssembly

@bluesign already started working on it 🎉 👏 See onflow/flow-emulator#301

With onflow/flow-emulator#301 merged, GOARCH=wasm GOOS=js go build ./cmd/emulator still fails, because flow-go itself does not build for WebAssembly:

Click to expand

$ GOARCH=wasm GOOS=js go build ./cmd/emulator
package github.com/onflow/flow-emulator/cmd/emulator
	imports github.com/onflow/flow-emulator
	imports github.com/onflow/flow-emulator/convert
	imports github.com/onflow/flow-emulator/convert/sdk
	imports github.com/onflow/flow-go/access
	imports github.com/onflow/flow-go/consensus/hotstuff/signature
	imports github.com/onflow/flow-go/storage
	imports github.com/dgraph-io/badger/v2
	imports golang.org/x/sys/unix: build constraints exclude all Go files in /Users/bastian/go/pkg/mod/golang.org/x/sys@v0.4.0/unix
package github.com/onflow/flow-emulator/cmd/emulator
	imports github.com/onflow/flow-emulator
	imports github.com/onflow/flow-emulator/storage/sqlite
	imports github.com/glebarez/go-sqlite
	imports modernc.org/libc
	imports modernc.org/libc/errno: build constraints exclude all Go files in /Users/bastian/go/pkg/mod/modernc.org/libc@v1.22.2/errno
package github.com/onflow/flow-emulator/cmd/emulator
	imports github.com/onflow/flow-emulator
	imports github.com/onflow/flow-emulator/storage/sqlite
	imports github.com/glebarez/go-sqlite
	imports modernc.org/libc
	imports modernc.org/libc/pthread: build constraints exclude all Go files in /Users/bastian/go/pkg/mod/modernc.org/libc@v1.22.2/pthread
package github.com/onflow/flow-emulator/cmd/emulator
	imports github.com/onflow/flow-emulator
	imports github.com/onflow/flow-emulator/storage/sqlite
	imports github.com/glebarez/go-sqlite
	imports modernc.org/libc
	imports modernc.org/libc/signal: build constraints exclude all Go files in /Users/bastian/go/pkg/mod/modernc.org/libc@v1.22.2/signal
package github.com/onflow/flow-emulator/cmd/emulator
	imports github.com/onflow/flow-emulator
	imports github.com/onflow/flow-emulator/storage/sqlite
	imports github.com/glebarez/go-sqlite
	imports modernc.org/libc
	imports modernc.org/libc/stdio: build constraints exclude all Go files in /Users/bastian/go/pkg/mod/modernc.org/libc@v1.22.2/stdio
package github.com/onflow/flow-emulator/cmd/emulator
	imports github.com/onflow/flow-emulator
	imports github.com/onflow/flow-emulator/storage/sqlite
	imports github.com/glebarez/go-sqlite
	imports modernc.org/libc/sys/types: build constraints exclude all Go files in /Users/bastian/go/pkg/mod/modernc.org/libc@v1.22.2/sys/types
package github.com/onflow/flow-emulator/cmd/emulator
	imports github.com/onflow/flow-emulator
	imports github.com/onflow/flow-emulator/storage/sqlite
	imports github.com/glebarez/go-sqlite
	imports modernc.org/libc
	imports modernc.org/libc/time: build constraints exclude all Go files in /Users/bastian/go/pkg/mod/modernc.org/libc@v1.22.2/time
package github.com/onflow/flow-emulator/cmd/emulator
	imports github.com/onflow/flow-emulator
	imports github.com/onflow/flow-emulator/storage/sqlite
	imports github.com/glebarez/go-sqlite
	imports modernc.org/libc
	imports modernc.org/libc/unistd: build constraints exclude all Go files in /Users/bastian/go/pkg/mod/modernc.org/libc@v1.22.2/unistd

@bluesign is also already trying to get flow-go to compile for WebAssembly 🎉 👏 See onflow/flow-go#3879

Remove test framework from flowkit, move further down in the dependency chain (Flow CLI)

Preventing regression

In flow-cli: We should build flowkit for WebAssembly on CI
In cadence-tools: We should build the LS for WebAssembly on CI, by running make wasm

Answer 2 · 2023-02-10T21:22:44.000Z

@turbolent there is an ugly workaround, if you wish ( while waiting flow-go fixes to land, it will take a bit of time probably )

replacing badger with https://github.com/bluesign/fakebadger in go.mod
for modernc ( sqlite ) part, I have another PR: onflow/flow-emulator#316 ( I was expecting it to compile to webassembly, but turned out it doesn't support too, but this PR uses old memory db )

Answer 3 · 2023-02-10T21:32:58.000Z

@bluesign Great idea to replace the problematic dependency (badger) with a mock/fake 👍
Also, thank you for working on bringing back a storage solution suitable for WebAssembly 👍

I'll ask the protocol team to give some guidance so timing/effort regarding replacing/abstracting badger in flow-go, i.e. if we should use the workaround or rather wait for the proper fix.

Answer 4 · 2023-02-13T11:30:03.000Z

I refactored flowkit to not depend on cadence-tools, I think generally speaking that's better architecture since any used tools should be used by the flow-cli package directly and not by its flowkit package.
onflow/flow-cli#845

Answer 5 · 2023-02-13T15:47:38.000Z

However, this is not proving to work. When I try to bisect and get to that same difference @turbolent you mentioned I manage to make it work if the /linter import changes. This is really weird. https://github.com/onflow/cadence-tools/compare/f46efb5..4e04255#diff-9dc030944615b63c919527ae35f799e2c63ff5137f18ef2165d07d808fd2b81fL52

Answer 6 · 2023-02-13T15:57:07.000Z

Also related: #32

Answer 7 · 2023-02-13T16:59:02.000Z

I believe the issue is coming out of cadence-tools using the flowkit/gateway which in turn uses emulator, which wasn't the case before. I can refactor so LS doesn't use the emulator and then for lint I believe I can do the same.

Answer 8 · 2023-02-13T17:46:11.000Z

@sideninja Sounds good! Thanks for the refactoring / decoupling. Please let me know if you need any help

Answer 9 · 2023-03-14T23:13:52.000Z

@sideninja Is the refactoring/decoupling happening in another issue or is this it?

Answer 10 · 2023-03-15T08:41:59.000Z

The work I'm doing refactoring flowkit now will allow the work to unblock this.