dora: A Rust repository from EricLBuehler

Website | Python API | Rust API | Guide | Discord

An extremely fast and simple dataflow oriented robotic framework to manage your projects and run complex apps, written in Rust.

Latency benchmark with Python API for both framework, sending 40M of random bytes.

Highlights

🚀 A single CLI to launch your Python and Rust robotic projects.
⚡️ 10-17x faster than ros2.
🐍 Easy and Fast prototyping with a clean Python API.
🖥️ Supports macOS, Linux, and Windows.
⏬ Installable without Rust via curl or powershell.
❇️ Includes a large set of pre-packaged nodes for fast prototyping.
🛠️ Build and Run applications without compilation step beyond the native compiler of your favourite language.
🤖 Simplifies building robotic applications by integrating hardware, algorithms, and AI models to facilitate seamless communication.
⚙️ Eases integration of hardware and software by supporting Python, C, C++, and ROS2, while ensuring low-latency communication with zero-copy Arrow messages.

Installation

Install dora with our standalone installers, or from crates.io:

On macOS and Linux

curl --proto '=https' --tlsv1.2 -sSf https://raw.githubusercontent.com/dora-rs/dora/main/install.sh | bash

On Windows

powershell -c "irm https://raw.githubusercontent.com/dora-rs/dora/main/install.ps1 | iex"

With cargo

cargo install dora-cli

Documentation

The full documentation is available on our website. A lot of guides are available on this section of our website.

Getting Started

Run the benchmark example (Rust must be installed):

git clone https://github.com/dora-rs/dora
cd dora
cargo run --example benchmark --release

Run some Python examples (A venv must be activated):

# activate your venv here
cd dora/examples/python-dataflow
dora up
dora build dataflow.yml
dora start dataflow.yml

Make sure to have a webcam

To stop your dataflow, you can use ctrl+c

What is Dora? And what features does Dora offer?

Dataflow-Oriented Robotic Architecture (dora-rs) is a framework that makes creation of robotic applications fast and simple.

dora-rs implements a declarative dataflow paradigm where tasks are split between nodes isolated as individual processes.

Each node defines its inputs and outputs to connect with other nodes.

nodes:
  - id: camera
    path: opencv-video-capture
    inputs:
      tick: dora/timer/millis/20
    outputs:
      - image
      -
  - id: plot
    path: opencv-plot
    inputs:
      image: camera/image

The dataflow paradigm has the advantage of creating an abstraction layer that makes robotic applications modular and easily configurable.

It offers several features, such as:

TCP Communication and Shared Memory

Communication between nodes is handled with shared memory on a same machine and TCP on distributed machines. Our shared memory implementation tracks messages across processes and discards them when obsolete. Shared memory slots are cached to avoid new memory allocation.

Arrow Message Format

Nodes communicate with Apache Arrow Data Format.

Apache Arrow is a universal memory format for flat and hierarchical data. The Arrow memory format supports zero-copy reads for lightning-fast data access without serialization overhead. It defines a C data interface without any build-time or link-time dependency requirement, that means that dora-rs has no compilation step beyond the native compiler of your favourite language.

Opentelemetry

dora-rs uses Opentelemetry to record all your logs, metrics and traces. This means that the data and telemetry can be linked using a shared abstraction.

Opentelemetry is an open source observability standard that makes dora-rs telemetry collectable by most backends such as elasticsearch, prometheus, Datadog...

Opentelemetry is language independent, backend agnostic, and easily collect distributed data, making it perfect for dora-rs applications.

Hot-Reloading

dora-rs implements Hot-Reloading for python which means you can change code at runtime in Python while keeping your state intact.

Using the feature flag: --attach --hot-reload, dora-rs watch for code change and reload nodes that has been modified.

You can check fail-safe mechanism at: dora-rs#239.

See this demo.

ROS2 Bridge

Note: this feature is marked as unstable.

Compilation Free Message passing to ROS 2
Automatic conversion ROS 2 Message <-> Arrow Array

import pyarrow as pa

# Configuration Boilerplate...
turtle_twist_writer = ...

## Arrow Based ROS2 Twist Message
## which does not require ROS2 import
message = pa.array([{
            "linear": {
                "x": 1,
            },
            "angular": {
                "z": 1
            },
        }])

turtle_twist_writer.publish(message)

You might want to use ChatGPT to write the Arrow Formatting: https://chat.openai.com/share/4eec1c6d-dbd2-46dc-b6cd-310d2895ba15

Showcases

Self-Coding Robot: Code RAG (WIP)

You can easily create a self-coding robot, by combining Hot-reloading with a Retrieval Augmented Generation (RAG) that is going to generate code modification from your prompt. See:examples/python-operator-dataflow

Self-Coding Robot is just the tip of the iceberg of robotics combined with llm, that we hope to power. There is so much more that we haven't explored yet like:

Cool Hardwares

Cool hardware that we think might be good fit to try out dora-rs 🙋 We are not sponsored by manufacturers:

	Price	Open Source	Github	type	Dora Project
DJI Robomaster S1	550$	SDK	https://github.com/dji-sdk/RoboMaster-SDK	Rover	https://huggingface.co/datasets/dora-rs/dora-robomaster
DJI Robomaster EP Core	950$	SDK	https://github.com/dji-sdk/RoboMaster-SDK	Rover, Arm
DJI Tello	100$			Drone
BitCraze Crazyflies	225$	Firmware, Lib, SDK	https://github.com/bitcraze	Drone
AlexanderKoch-Koch/low_cost_robot	250$	Everything	https://github.com/AlexanderKoch-Koch/low_cost_robot	Arm
xArm 1S	200$			Arm
Wavego	250$			Quadruplet
AINex	800$			Humanoid

For more: https://docs.google.com/spreadsheets/d/1YYeW2jfOIWDVgdEgqnMvltonHquQ7K8OZCrnJRELL6o/edit#gid=0

Support Matrix

	dora-rs	Hoped for
Tier 1 Support	Python, Rust	C, C++, ROS 2
Tier 2 Support	C, C++, ROS2
Hot-reloading	Python	Rust (https://github.com/orgs/dora-rs/discussions/360)
Message Format	Arrow	Native
Local Communication	Shared Memory	Custom Middleware, zero-copy GPU IPC, intra-process `tokio::channel` communication
Remote Communication	TCP	Custom Middleware, Zenoh
Metrics, Tracing, and Logging	Opentelemetry	Native logging libraries into Opentelemetry
Data archives	Parquet (dora-record)
Visualization and annotation	OpenCV	rerun.io
Supported Platforms (x86)	Windows, macOS, Linux
Supported Platforms (ARM)	macOS, Linux
Configuration	YAML

Contributing

We are passionate about supporting contributors of all levels of experience and would love to see you get involved in the project. See the contributing guide to get started.

Discussions

Our main communication channels are:

Feel free to reach out on any topic, issues or ideas.

We also have a contributing guide.

License

This project is licensed under Apache-2.0. Check out NOTICE.md for more information.

EricLBuehler/dora