Malloy Composer Demo
The Malloy Composer Demo is provided as a working example of an application built on top of Malloy. If you have any questions about getting it running, please reach out to us for help! If you find bugs or have feature requests, you can submit them as issues in this repo. Learn how to use the composer
Malloy Composer can be run with a built in web server (Full Mode) and can access BigQuery, Postgres and DuckDB or in WASM Mode, running entirely in a web browser. In WASM Mode, Malloy Composer can only access csv and parquet files (via DuckDB).
The composer is only intended for demo purposes, and is not a finished or supported product.
GitHub mutes videos by default, so make sure to unmute.
4k.mov
Downloading and Running Composer (Full Mode)
- Download a Release of Composer and Sample models
- Unzip the archive.
- For MacOS, Linux:
- Make sure the binary is executable:
chmod a+x ./composer
- Remove the Apple quarantine flag:
xattr -d com.apple.quarantine ./composer
- Make sure the binary is executable:
- In your newly unzipped directory, run
./composer malloy-samples
Downloading and Running WASM Composer
Composer can run from a web server and is included in the samples. WASM Composer can only use csv and parquet files (via DuckDB).
- Run the following commands
git clone git@github.com:malloydata/malloy-samples.git
cd malloy-samples
python -m SimpleHTTPServer
- Goto http://localhost:8000/wasm in a web browser
Running Composer from Source
npm install
to install package dependenciesgit submodule init
andgit submodule update
to install git dependencies 4npm run build
npm run start malloy-samples
Database Connections
You don't need to setup anything to run the DuckDB examples (the data is included with the samples).
To run the BigQuery Samples uou will need to have a Google Cloud Account, access to BigQuery, and the gcloud CLI installed. Once the gcloud CLI is installed, open a terminal and type the following:
gcloud auth login --update-adc
gcloud config set project {my_project_id} --installation
Replace {my_project_id} with the ID of the BigQuery project you want to use & bill to. If you're not sure what this ID is, open Cloud Console, and click on the dropdown at the top (just to the right of the "Google Cloud Platform" text) to view projects you have access to. If you don't already have a project, create one.
VSCode
You will likely want to set up the VS Code Extension to view and edit Malloy files.
Composer Parameters
This will start local webserver at http://localhost:4000 by default. The path
argument allows you to configure what Malloy files you want the composer to have access to. There are several different options for what the path
can be.
- A path to a single
.malloy
file - A path to a directory containing multiple
.malloy
files* - A path to a dataset config
.json
file** - A path to an app config
.json
file**, specifying apath
for multiple datasets. These dataset-specific paths can be any above path types.
*Note that if you pass a directory containing a composer.json
file, it will be used as either a "dataset" config .json
file** or an "app" config .json
file** depending on its contents.
**See below for example config files
If you don't have a particular dataset you want to try out, you can always use the sample models: npm run start ./malloy-samples
. You can also view these sample models in an online version of the composer.
Troubleshooting Notes
- If your datasets are visible, but clicking on one doesn't work, check in the output of the
npm run start
command for errors. One of your Malloy files or config files may have an error in it. - You'll need to define a source for it to be explorable; top-level named queries that are not inside a source are not explorable.
Config File Examples
"Dataset" config .json
file:
{
"readme": <path to a readme file>,
"title": <title of dataset>,
"models": [
{
"id": "flights",
"path": <path to a .malloy file>,
"sources": [
{
"title": <title of source>,
"sourceName": <name of source in the .malloy file>,
"description": "Look at all the flights that have flown up to 2003"
},
...
]
}
]
}
"App" config .json
file:
{
"apps": [
{
"id": "faa",
"path": <path to dataset>
},
...
],
"readme": <path to readme file>
}
Development
Running against a local version of Malloy
If you're working on developing Malloy, and have changes to one of the other Malloy libraries locally on your machine, you can link the Composer to that version.
- In your local Malloy repository, run
npm link -ws
. That will make your development packages locally available for development. - In your VS Code extension repository, run
npm run malloy-link
to use your local Malloy packages. - If you make changes to Malloy that are required by the extension, merges those into main, and that will trigger an automatic developer release of Malloy.
- Once that release completes, run
npm run malloy-update
to update dependencies to that release. This will break the link to your local version of Malloy, so if you want to resume local development, re-runnpm run malloy-link
- To manually unlink without updating, you may run
npm run malloy-unlink
Debugging
Using VSCode
- From VSCode Run & Debug Panel Select "Launch Composer" from the dropdown, then "Start Debugging" using the Run button or
F5
- Then select "Launch Composer" from the dropdown, then "Start Debugging" using the Run button or
F5
- To connect to the render process select "Attach to Composer Render Process"
Running Composer entirely in Browser Mode (using DuckDB WASM)
Composer can run entirely in a browser with an included SQL engine. Data and the composer applicaition is servered from a vanilla web server (See lloyd's blogpost on the subject).
There is a duckdb-wasm
folder which should be served on a webserver in order to run an instance of the DuckDB WASM Mode Composer. The duckdb-wasm/dist
folder contains the compiled app after running npm run build-duckdb-wasm
. You can also run npm run start-duckdb-wasm
to run the DuckDB WASM Mode Composer on port 9999.
The WASM Mode Composer can be easily added to any project that already uses the Fiddle, or it can be added to any repo with Malloy files using CSV or parquet files to simply serve the Composer experience on top of those files. All you need is to host the generated app.js
and app.css
files on some CDN (it can be the same CDN that serves the data and malloy files, or a different one), then to add a composer.json
file that lists the available datasets (see the "app" config .json
file example above).
Finally, add an index.html
file, like this:
<!DOCTYPE html>
<html lang="en">
<head>
<title>Malloy Composer</title>
<link rel="stylesheet" href="dist/app.css" />
<script src="dist/app.js" async defer></script>
</head>
<body>
<noscript>You need to enable JavaScript to run this app.</noscript>
<div id="root"></div>
</body>
</html>