/hpcc-js-wasm

HPCC-Systems Web-Assembly (JavaScript)

Primary LanguageTypeScriptApache License 2.0Apache-2.0

@hpcc-js/wasm

Test PR

This repository contains a collection of useful c++ libraries compiled to WASM for (re)use in Node JS, Web Browsers and JavaScript Libraries:

Built with:

Quick GraphViz Demos

Installation

The simplest way to include this project is via NPM:

npm install --save @hpcc-js/wasm

CLI

To call dot-wasm without installing:

npx -p @hpcc-js/wasm dot-wasm [file | 'dot']

To install the global command dot-wasm via NPM:

npm install --global @hpcc-js/wasm

Usage:

Usage: dot-wasm [options] fileOrDot

Options:
      --version  Show version number                                   [boolean]
  -K, --layout   Set layout engine (circo | dot | fdp | sfdp | neato | osage |
                 patchwork | twopi). By default, dot is used.
  -T, --format   Set output language to one of the supported formats (svg, dot,
                 json, dot_json, xdot_json, plain, plain-ext). By default, svg
                 is produced.
  -h, --help     Show help                                             [boolean]

Examples:
  dot-wasm -K neato -T xdot ./input.dot  Execute NEATO layout and outputs XDOT
                                         format.

Contents

@hpcc-js/wasm includes the following files in its dist folder:

  • index.js / index.min.js files: Exposes all the available APIs for all WASM files.
  • WASM Files:
    • graphvizlib.wasm
    • expatlib.wasm
    • ...more to follow...

Important: WASM files are dynamically loaded at runtime (this is a browser / emscripten requirement), which has a few implications for the consumer:

Pros:

  • While this package has potentially many large WASM files, only the ones being used will ever be downloaded from your CDN / Web Server.

Cons:

  • Most browsers don't support fetch and loading pages via file:// URN, so for testing / development work you will need to run a test web server.
  • Bundlers (RollupJS / WebPack) will ignore the WASM files, so you will need to manually ensure they are present in your final distribution (typically they are placed in the same folder as the bundled JS)

API Reference

Common

Utility functions relating to @hpcc-js/wasm as a package

# wasmFolder([url]) · <>

If url is specified, sets the default location for all WASM files. If url is not specified it returns the current url (defaults to undefined).

# __hpcc_wasmFolder · <>

Global variable for setting default WASM location, this is an alternative to wasmFolder

GraphViz (graphvizlib.wasm)

GraphViz WASM library, see graphviz.org for c++ details. While this package is similar to Viz.js, it employs a completely different build methodology taken from GraphControl.

The GraphViz library comes in two flavours

  • An exported graphviz namespace, where each API function is asynchrounous and returns a Promise<string>.
  • A graphvizSync asynchrounous function which returns a Promise<GraphvizSync> which is a mirror instance of graphviz, where each API function is synchrounous and returns a string.

Hello World

<!DOCTYPE html>
<html>

<head>
    <meta charset="UTF-8">
    <title>GraphViz WASM</title>
    <script src="https://cdn.jsdelivr.net/npm/@hpcc-js/wasm/dist/index.min.js"></script>
    <script>
        var hpccWasm = window["@hpcc-js/wasm"];
    </script>
</head>

<body>
    <div id="placeholder"></div>
    <div id="placeholder2"></div>
    <script>
        const dot = `
            digraph G {
                node [shape=rect];

                subgraph cluster_0 {
                    style=filled;
                    color=lightgrey;
                    node [style=filled,color=white];
                    a0 -> a1 -> a2 -> a3;
                    label = "Hello";
                }

                subgraph cluster_1 {
                    node [style=filled];
                    b0 -> b1 -> b2 -> b3;
                    label = "World";
                    color=blue
                }

                start -> a0;
                start -> b0;
                a1 -> b3;
                b2 -> a3;
                a3 -> a0;
                a3 -> end;
                b3 -> end;

                start [shape=Mdiamond];
                end [shape=Msquare];
            }
        `;

        // Asynchronous call to layout
        hpccWasm.graphviz.layout(dot, "svg", "dot").then(svg => {
            const div = document.getElementById("placeholder");
            div.innerHTML = svg;
        });

        hpccWasm.graphvizSync().then(graphviz => {
            const div = document.getElementById("placeholder2");
            // Synchronous call to layout
            div.innerHTML = graphviz.layout(dot, "svg", "dot");
        });
    </script>

</body>

</html>

GraphViz API

# graphvizVersion() · <>

Returns the Graphviz Version.

# layout(dotSource[, outputFormat][, layoutEngine][, ext]) · <>

Performs layout for the supplied dotSource, see The DOT Language for specification.

outputFormat supports the following options:

  • dot
  • dot_json
  • json
  • svg (default)
  • xdot_json

See Output Formats for more information.

layoutEngine supports the following options:

  • circo
  • dot (default)
  • fdp
  • sfdp
  • neato
  • osage
  • patchwork
  • twopi

See Layout manual pages for more information.

ext optional "extra params":

  • images: An optional array of
{
    path: string;   //  The path for the image.
    width: string;  //  Width of Image
    height: string; //  Height of Image
}
  • files: An optional array of
{
    path: string;   //  The path for the file.
    data: string;   //  The data for the file.
}
  • wasmFolder: An optional string specifying the location of wasm file.
  • wasmBinary: An optional "pre-fetched" copy of the wasm binary as returned from XHR or fetch.
  • yInvert: An optional boolean flag to invert the y coordinate in generic output formats (dot, xdot, plain, plain-ext). This is equivalent to specifying -y when invoking Graphviz from the command-line.
  • nop: An optional number to specify "No layout" mode for the neato engine. This is equivalent to specifying the -n option when invoking Graphviz from the command-line.

For example passing a web hosted Image to GraphViz:

hpccWasm.graphviz.layout('digraph { a[image="https://.../image.png"]; }', "svg", "dot", { 
    images: [{ 
        path: "https://.../image.png", 
        width: "272px", 
        height: "92px" 
    }] 
}).then(svg => {
    document.getElementById("placeholder").innerHTML = svg;
}).catch(err => console.error(err.message));

# circo(dotSource[, outputFormat][, ext]) · <>

Convenience function that performs circo layout, is equivalent to layout(dotSource, outputFormat, "circo");.

# dot(dotSource[, outputFormat][, ext]) · <>

Convenience function that performs dot layout, is equivalent to layout(dotSource, outputFormat, "dot");.

# fdp(dotSource[, outputFormat][, ext]) · <>

Convenience function that performs fdp layout, is equivalent to layout(dotSource, outputFormat, "fdp");.

# sfdp(dotSource[, outputFormat][, ext]) · <>

Convenience function that performs sfdp layout, is equivalent to layout(dotSource, outputFormat, "sfdp");.

# neato(dotSource[, outputFormat][, ext]) · <>

Convenience function that performs neato layout, is equivalent to layout(dotSource, outputFormat, "neato");.

# osage(dotSource[, outputFormat][, ext]) · <>

Convenience function that performs osage layout, is equivalent to layout(dotSource, outputFormat, "osage");.

# patchwork(dotSource[, outputFormat][, ext]) · <>

Convenience function that performs patchwork layout, is equivalent to layout(dotSource, outputFormat, "patchwork");.

# twopi(dotSource[, outputFormat][, ext]) · <>

Convenience function that performs twopi layout, is equivalent to layout(dotSource, outputFormat, "twopi");.

# graphvizSync([wasmFolder], [wasmBinary]) · <>

Returns a Promise<GraphvizSync>, once resolved provides a synchronous variant of the above methods. Has an optional wasmFolder argument to override the default wasmFolder location and optional wasmBinary to short circuit the wasm downloading process.

Expat (expatlib.wasm)

Expat WASM library, provides a simplified wrapper around the Expat XML Parser library, see libexpat.github.io for c++ details.

Hello World

<!DOCTYPE html>
<html>

<head>
    <meta charset="UTF-8">
    <title>GraphViz WASM</title>
    <script src="https://unpkg.com/@hpcc-js/wasm/dist/index.min.js"></script>
    <script>
        var hpccWasm = window["@hpcc-js/wasm"];
    </script>
</head>

<body>
    <script>
        const xml = `
            <root>
                <child xxx="yyy">content</child>
            </root>
        `;

        var callback = {
            startElement(tag, attrs) { console.log("start", tag, attrs); },
            endElement(tag) { console.log("end", tag); },
            characterData(content) { console.log("characterData", content); }
        };
        hpccWasm.parse(xml, callback);
    </script>

</body>

</html>

Expat API

# expatVersion() · <>

Returns the Expat Version.

# parse(xml, callback) · <>

  • xml: XML String.
  • callback: Callback Object with the following methods:
    • startElement(tag: string, attrs: {[key: string]: string}): void;
    • endElement(tag: string): void;
    • characterData(content: string): void;

Parses the XML with suitable callbacks.

Note: characterData may get called several times for a single tag element.

Building @hpcc-js/wasm

Building is supported on both Linux (tested with Ubuntu 20.04) and Windows with WSL enabled (Ubuntu-20.04). Building in other environments should work, but may be missing certain prerequisites.

These are then known required OS dependencies (see ./docker/ubuntu-dev.dockerfile for test script):

sudo apt-get install -y curl
sudo curl --silent --location https://deb.nodesource.com/setup_14.x | sudo bash -
sudo apt-get install -y nodejs
sudo apt-get install -y build-essential

sudo apt-get install -y git cmake wget
sudo apt-get install -y gcc-multilib g++-multilib pkg-config autoconf bison libtool flex zlib1g-dev 
sudo apt-get install -y python3 python3-pip

Build steps:

git clone https://github.com/hpcc-systems/hpcc-js-wasm.git
cd hpcc-js-wasm
npm ci
npm run install-build-deps
npm run build

Note: The install-build-deps downloads the following dependencies:

This has been made a manual step as the downloads are quite large and the auto-configuration can be time consuming.

Clean dependencies:

It is worth noting that npm run clean will only clean any artifacts associated with the build, but won't clean clean any of the third party dependencies. To remove those for a "full clean", run:

npm run clean-build-deps