WebAssembly WebTerm

Run your WebAssembly binaries on a terminal/tty emulation in your browser. Emscripten and WASI are supported. This project is developed as an addon for xterm.js v4, so you can easily use it in your own projects.

It originated from the CrypTool project in 2022 for running OpenSSL v3 in a browser.

Please note that xterm.js and this addon need a browser to run.

Readme Contents

Installation and Usage (via script tag, Node.js, or React)
Binaries (Predelivery, Compiling C/C++, and Compiling Rust)
Internal workings, JS commands, and WasmWebTerm.js Code API
Contributing and License

Installation

First, install Node.js and npm. Then install xterm.js and wasm-webterm:

npm install xterm cryptool-org/wasm-webterm --save

Usage

JavaScript can be written for browsers or nodes, but this addon needs a browser to run (or at least a DOM and Workers or a window object). So if you use Node.js, you have to also use a web bundler like Webpack or Parcel. Using plain JS does not require a bundler.

Please note: To make use of WebWorkers you will need to configure your server or web bundler to use custom HTTPS headers for cross-origin isolation. You can find an example using Webpack in the examples folder.

Choose the variant that works best for your existing setup:

Variant 1: Load via plain JS `<script>` tag

The first and most simple way is to include the prebundled webterm.bundle.js into an HTML page using a <script> tag.

Create an HTML file (let's say index.html) and open it in your browser. You could also use example 1 in the examples folder.

<html>
    <head>
        <script src="node_modules/xterm/lib/xterm.js"></script>
        <link rel="stylesheet" href="node_modules/xterm/css/xterm.css"/>
        <script src="node_modules/wasm-webterm/webterm.bundle.js"></script>
    </head>
    <body>

        <div id="terminal"></div>

        <script>
            let term = new Terminal()
            term.loadAddon(new WasmWebTerm.default())
            term.open(document.getElementById("terminal"))
        </script>

        <style>
            html, body { margin: 0; padding: 0; background: #000; }
            .xterm.terminal { height: calc(100vh - 2rem); padding: 1rem; }
        </style>
    </body>
</html>

Please note that the plain JS version uses new WasmWebTerm.default() [containing .default] instead of just new WasmWebTerm() like in the Node.js examples.

Variant 2: Import as Node.js module and use a web bundler

If you are writing a Node.js module and use a web bundler to make it runnable in web browsers, here's how you could include this project:

You can also see example 2 in the examples folder. We used Parcel as an example, but any other bundler would work too.

Create a JS file (let's say index.js)

import { Terminal } from "xterm"
import WasmWebTerm from "wasm-webterm"

let term = new Terminal()
term.loadAddon(new WasmWebTerm())
term.open(document.getElementById("terminal"))

Create an HTML file (let's say index.html)

<html>
    <head>
        <link rel="stylesheet" href="node_modules/xterm/css/xterm.css" />
    </head>
      <body>
          <div id="terminal"></div>
          <script src="./index.js" type="module"></script>

          <style>
              html, body { margin: 0; padding: 0; background: #000; }
              .xterm.terminal { height: calc(100vh - 2rem); padding: 1rem; }
          </style>
      </body>
</html>

Use a web bundler to make it run in a browser

npm install -g parcel-bundler
parcel index.html

Variant 3: Using React and a web bundler

If you are using React, example 3 in the examples folder includes a React wrapper for xterm.js that was taken from xterm-for-react. We can use this to pass our addon.

The following code is not complete (you'd also need an HTML spawnpoint and a web bundler like Webpack) and we recommend to see the React example.

import ReactDOM from "react-dom"
import XTerm from "./examples/3-react-with-webpack/xterm-for-react"
import WasmWebTerm from "wasm-webterm"

ReactDOM.render(<XTerm addons={[new WasmWebTerm()]} />,
    document.getElementById("terminal"))

Binaries

This addon executes WebAssembly binaries. They are compiled from native languages like C, C++, Rust, etc.

WebAssembly binaries are files ending on .wasm and can either be predelivered by you (shipping them with your application) or added live via drag and drop by users. If no binary was found locally, wapm.io is fetched.

What is a runtime and why do we need it?

"WebAssembly is an assembly language for a conceptual machine, not a physical one. This is why it can be run across a variety of different machine architectures." (source)

To run programs intended to run in an OS like Linux, the "machine architecture" (your browser which is running JS) needs to initialize a runtime environment. It provides a virtual memory filesystem, handles system-calls, etc.

When using WASI (a standard) this is handled by WASI from wasmer-js v0.12. You can alternatively use compilers like Emscripten, which will generate a specific .js file containing the JS runtime for your wasm binary.

If you provide a .js file with the same name than your .wasm file (for example drop or ship test.wasm and test.js together), the .wasm binary will be interpreted as compiled with Emscripten and use the .js file as its runtime. If you just drop a .wasm file, it's interpreted as WASI.

Predelivering binaries

When you host your webterm instance somewhere, you might want to deliver some precompiled wasm binaries for your users to use. For example, we compiled OpenSSL with Emscripten to run it in the webterm.

See below how to compile them. Then copy your binaries (.wasm and optionally .js files) into a folder, let's say ./binaries. Make sure, that your web bundler (or however you're serving your project) also delivers these binaries, so that they're available when running the webterm. We used Webpack's CopyPlugin in our React example.

Then pass their path to the WasmWebTerm instance:

let wasmterm = new WasmWebTerm("./binaries")

When executing a command on the webterm, it will fetch <binarypath>/<programname>.wasm and validate if it's WebAssembly. So make sure, that the file name of your wasm binary matches the command name. If it's available, it'll also try to fetch <binarypath>/<programname>.js and thereby determine if WASI or Emscripten.

Compiling C/C++ to `.wasm` binaries

C or C++ code can be compiled to WebAssembly using Emscripten or a WASI compliant compiler like WASI CC.

In both following examples we will use this little C program and put it in a file named test.c.

#include <stdio.h>

int main()
{
    char name[200];
    fgets(name, 200, stdin);
    printf("You entered: %s", name);
    return 0;
}

Example 1: Compile with Emscripten

First, install the Emscripten SDK. It supplies emcc and tools like emconfigure and emmake for building projects.

Running the following command will create two files: test.wasm (containing the WebAssembly binary) and test.js (containing a JS runtime for that specific wasm binary). The flags are used to configure the JS runtime:

$ emcc test.c -o test.js -s EXPORT_NAME='EmscrJSR_test' -s ENVIRONMENT=web,worker -s FILESYSTEM=1 -s MODULARIZE=1 -s EXPORTED_RUNTIME_METHODS=callMain,FS,TTY -s INVOKE_RUN=0 -s EXIT_RUNTIME=1 -s EXPORT_ES6=0 -s USE_ES6_IMPORT_META=0 -s ALLOW_MEMORY_GROWTH=1

Explain these flags to me

You can also use other Emscripten flags, as long as they don't interfere with the flags we've used here. These are essential. Here's what they mean:

Flag	Value	Description
EXPORT_NAME	EmscrJSR_<programname>	FIXED name for Module, needs to match exactly to work!
ENVIRONMENT	web,worker	Specifies we don't need Node.js (only web and worker)
FILESYSTEM	1	Make sure Emscripten inits a memory filesystem (MemFS)
MODULARIZE	1	Use a Module factory so we can create custom instances
EXPORTED_RUNTIME_METHODS	callMain,FS,TTY	Export Filesystem, Teletypewriter, and our main method
INVOKE_RUN	0	Do not run immediatly when instanciated (but manually)
EXIT_RUNTIME	1	Exit JS runtime after wasm, will be re-init by webterm
EXPORT_ES6	0	Do not export as ES6 module so we can load in browser
USE_ES6_IMPORT_META	0	Also do not import via ES6 to easily run in a browser
ALLOW_MEMORY_GROWTH	1	Allow the memory to grow (allocate more memory space)

ℹ️ The fixed Emscripten Module name is a todo! If you have ideas for an elegant solution, please let us now :)

Then copy the created files test.wasm and test.js into your predelivery folder or drag&drop them into the terminal window. You can now execute the command "test" in the terminal and it should ask you for input.

Example 2: Compile with WASI CC

First, install wasienv. It includes wasicc and tools like wasiconfigure and wasimake.

You can then compile test.c with the following line:

$ wasicc test.c -o test.wasm

There is no need for lots of flags here, because WASI is a standard interface and uses a standardized JS runtime for all binaries.

Then copy the created file test.wasm into your predelivery folder or drag&drop it into the terminal window. You can now execute the command "test" in the terminal and it should ask you for input.

Compiling Rust to `.wasm` binaries

Rust code can be compiled to target wasm32-wasi which can be executed by this addon. You can either compile it directly with rustc or by using Rust's build tool cargo.

If you haven't already, install Rust. Then install the wasm32-wasi target:

$ rustup target add wasm32-wasi

Example 1: Using `rustc`

Take some Rust source code, let's say in a file named test.rs

fn main() {
    println!("Hello, world!");
}

and compile it with

$ rustc test.rs --target wasm32-wasi

Then copy the created file test.wasm into your predelivery folder or drag&drop it into the terminal window. You can now execute the command "test" in the terminal and it should print Hello, world! to you.

Example 2: Using `cargo`

Create a new project

$ cargo new <projectname>
$ cd <projectname>

and build it to wasm32-wasi

$ cargo build --target=wasm32-wasi

You should find the binary <projectname>.wasm in the folder <projectname>/target/wasm32-wasi/debug.

Copy it into your predelivery folder or drag&drop it into the terminal window. You can now execute the command "<projectname>" in the terminal.

Internal procedure flow

When a user visits your page, it loads xterm.js and attaches our addon. See the upper code examples. That calls the xterm.js life cycle method activate(xterm) in WasmWebTerm.js which starts the REPL.

The REPL waits for the user to enter a line (any string, usually commands) into the terminal. This line is then evaluated by runLine(line). If there is a predefined JS command, it'll execute it. If not, it'll delegate to runWasmCommand(..) (or runWasmCommandHeadless(..) when piping).

This then calls _getOrFetchWasmModule(..). It will search for a WebAssembly binary with the name of the command in the predelivery folder. If none is found, it'll fetch wapm.io.

The binary will then be passed to an instance of WasmRunner. If it receives both a wasm binary and a JS runtime, it'll instanciate an EmscrWasmRunnable. If it only received a wasm binary, it'll instanciate a WasmerRunnable. Both runnables setup the runtime required for the wasm execution and start the execution.

If WebWorker support is available (including SharedArrayBuffers and Atomics), this will be wrapped into a Worker thread (see WasmWorker.js) using Comlink. This is done using a Blob instead of delivering a separate Worker JS file: When importing WasmWorker.js, Webpack will prebuild/bundle all its dependencies and return it as "asset/source" (plain text) instead of a instantiable class. This is done using a Webpack loader.

Communication between the WasmRunner and the xterm.js window is done trough Comlink proxy callbacks, as they might be on different threads. For example, if the wasm binary asks for Stdin (while running on the worker thread), it'll be paused, the Comlink proxy _stdinProxy is called, and the execution resumes after the proxy has finished.

This pausing on the worker thread is done by using Atomics. That's why we rely on that browser support. The fallback (prompts) pauses the browser main thread by calling window.prompt(), which also blocks execution.

When the execution has finished, the respective onFinish(..) callback is called and the REPL starts again.

`WasmWebTerm.js` Code API

The code API of the main class WasmWebterm is documented in src/WasmWebTerm.md.

Defining custom JS commands

In addition to running WebAssembly, you can also run JS commands on the terminal. You can register them with registerJsCommand(name, callback). When typing name into the terminal, the callback function is called.

The callback function will receive argv (array) and stdinPreset (string) as input parameters. Output can be returned, resolve()d or yielded.

todo: stderr and file system access are not implemented yet

Simple echo examples:

wasmterm.registerJsCommand("echo1", (argv) => {
    return argv.join(" ") // sync and normal return
})

wasmterm.registerJsCommand("echo2", async (argv) => {
    return argv.join(" ") // async function return
})

wasmterm.registerJsCommand("echo3", async (argv) => {
    return new Promise(resolve => resolve(argv.join(" "))) // promise resolve()
})

wasmterm.registerJsCommand("echo4", async function*(argv) {
    for(const char of argv.join(" ")) yield char // generator yield
})

Contributing

Any contributions are greatly appreciated. If you have a suggestion that would make this better, please open an issue or fork the repository and create a pull request.

License

Distributed under the Apache-2.0 License. See LICENSE for more information.

cryptool-org/wasm-webterm