MeshCat is a remotely-controllable 3D viewer, built on top of three.js. The viewer contains a tree of objects and transformations (i.e. a scene graph) and allows those objects and transformations to be added and manipulated with simple commands. This makes it easy to create 3D visualizations of geometries, mechanisms, and robots.
The MeshCat architecture is based on the model used by Jupyter:
- The viewer itself runs entirely in the browser, with no external dependencies
- The MeshCat server communicates with the viewer via WebSockets
- Your code can use the meshcat python libraries or communicate directly with the server through its ZeroMQ socket.
The latest version of MeshCat requires Python 3.6 or above.
Using pip:
pip install meshcat
From source:
git clone https://github.com/meshcat-dev/meshcat-python git submodule update --init --recursive cd meshcat-python python setup.py install
You will need the ZeroMQ libraries installed on your system:
Ubuntu/Debian:
apt install libzmq3-dev
Homebrew:
brew install zmq
Windows:
Download the official installer from zeromq.org.
For examples of interactive usage, see demo.ipynb
If you want to run your own meshcat server (for example, to communicate with the viewer over ZeroMQ from another language), all you need to do is run:
meshcat-server
The server will choose an available ZeroMQ URL and print that URL over stdout. If you want to specify a URL, just do:
meshcat-server --zmq-url=<your URL>
You can also instruct the server to open a browser window with:
meshcat-server --open
All communication with the meshcat server happens over the ZMQ socket. Some commands consist of multiple ZMQ frames.
| ZMQ frames: | ["url"] |
|---|---|
| Action: | Request URL |
| Response: | The web URL for the server. Open this URL in your browser to see the 3D scene. |
| ZMQ frames: | ["wait"] |
|---|---|
| Action: | Wait for a browser to connect |
| Response: | "ok" when a brower has connected to the server. This is useful in scripts to block execution until geometry can actually be displayed. |
| ZMQ frames: | ["set_object", "/slash/separated/path", data] |
|---|---|
| Action: | Set the object at the given path. data is a MsgPack-encoded dictionary, described below. |
| Response: | "ok" |
| ZMQ frames: | ["set_transform", "/slash/separated/path", data] |
|---|---|
| Action: | Set the transform of the object at the given path. There does not need to be any geometry at that path yet, so set_transform and set_object can happen in any order. data is a MsgPack-encoded dictionary, described below. |
| Response: | "ok" |
| ZMQ frames: | ["delete", "/slash/separated/path", data] |
|---|---|
| Action: | Delete the object at the given path. data is a MsgPack-encoded dictionary, described below. |
| Response: | "ok" |
{
"type": "set_object",
"path": "/slash/separated/path", // the path of the object
"object": <three.js JSON>
}
The format of the object field is exactly the built-in JSON serialization format from three.js (note that we use the JSON structure, but actually use msgpack for the encoding due to its much better performance). For examples of the JSON structure, see the three.js wiki .
- Note on redundancy
- The
typeandpathfields are duplicated: they are sent once in the first two ZeroMQ frames and once inside the MsgPack-encoded data. This is intentional and makes it easier for the server to handle messages without unpacking them fully.
{
"type": "set_transform",
"path": "/slash/separated/path",
"matrix": [1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1]
}
The format of the matrix in a set_transform command is a column-major homogeneous transformation matrix.
{
"type": "delete",
"path", "/slash/separated/path"
}
Creating a box at path /meshcat/box
{
"type": "set_object",
"path": "/meshcat/box",
"object": {
"metadata": {"type": "Object", "version": 4.5},
"geometries": [{"depth": 0.5,
"height": 0.5,
"type": "BoxGeometry",
"uuid": "fbafc3d6-18f8-11e8-b16e-f8b156fe4628",
"width": 0.5}],
"materials": [{"color": 16777215,
"reflectivity": 0.5,
"type": "MeshPhongMaterial",
"uuid": "e3c21698-18f8-11e8-b16e-f8b156fe4628"}],
"object": {"geometry": "fbafc3d6-18f8-11e8-b16e-f8b156fe4628",
"material": "e3c21698-18f8-11e8-b16e-f8b156fe4628",
"matrix": [1.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 1.0],
"type": "Mesh",
"uuid": "fbafc3d7-18f8-11e8-b16e-f8b156fe4628"}},
}
Translating that box by the vector [2, 3, 4]:
{
"type": "set_transform",
"path": "/meshcat/box",
"matrix": [1.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 2.0, 3.0, 4.0, 1.0]
}
Msgpack's default behavior is not ideal for packing large contiguous arrays (it inserts a type code before every element). For faster transfer of large pointclouds and meshes, msgpack Ext codes are available for several types of arrays. For the full list, see https://github.com/kawanet/msgpack-lite#extension-types . The meshcat Python bindings will automatically use these Ext types for numpy array inputs.