This repo houses a framework for interacting with and controlling iCreate3 robots using OSC or Open Sound Control. It also has a bridge between OSC and ROS2.
The OSC PIBOT framework goals are to provide a OSC based interface that allows for artist-friendly improvisational and performative control of a variety of robots, LEDs, motors and I2C devices from existing media and performance software tools like Max-MSP, Isadora, TouchDesigner, Qlab, and any other OSC capable software.
For each robot, a compute board such as a Raspberry Pi 4B, installed with Ubuntu 22.04, is running a Python script that sets up an OSC server, acting as a bridge between all other OSC clients and the iCreate3 itself (which can't receive OSC natively). Each Pi only listens for messages relating to its assigned robot, filters them, expands them to lower-level commands, and sends them directly to the robot via Bluetooth. For more reliable Bluetooth connection with longer range coverage, the Pis are mounted on the robots and receive OSC messages via a local WiFi network. An optional 'hub' computer may be employed to coordinate multiple robots.
Sample MAX-MSP and Isadora patches for use with the PIBOT framework are on the EXAMPLES page. OSC PIBOT can be used with any OSC sender / receiver and additional sample files are welcomed.
Minimum Requirements: Ubuntu 22.04 or similar OS on a Raspberry Pi (tested w/ a Pi 4B w/ 4G ram) connected to a robot via bluetooth, a robot (such as an ICreate3) based on irobot-edu-python-sdk, control machine with an OSC sending software source, local network allowing UDP communication between devices.
Two modes:
- peer to peer - nodes run on the Pi on each robot and each directly receive/ transmit OSC messages
- one client, multiple servers - a machine running a robot sequencer (written in Clojure) can receive cue sheets with various behavioral routines placed on a timeline and coordinate the bots accordingly.
An experimental (and currently incomplete) ROS2 backend exists under src/osc_bridge. Its development was temporarily halted in favor of the Bluetooth-based iRobot EDU Python SDK, which appeared to have much better latency of command execution. We hope for future development to include additional ROS2 research, and welcome contributions.
The framework provides unified OSC based control of supplemental I2C components controlled via the Pi. The customised boards developed provide 16 channels of I2C PWM based control of 12/24v LED lighting arrays as well as servo motor control, I2S audio outputs / input via an Adafruit MAX98357 I2S amplifier, and charge control for a supplemental USB 3 PD battery bank to deliver 12 and 15v power. The board is sized to fit in the iCreate drawer alongside a raspberry pi.
- A schematic file of the board we developed is available here. Please contact us if you would like information for a buildable copy of the board.
- I2S software support is in process.
The OSC PIBOT framework is intended for use on a raspberry Pi 4 or later running Ubuntu 22.04, and to communicate with an iCreate 3 or iCreate Root robot on the same local network as the Pi and OSC sender.
The Pi should be setup running Ubuntu 22.04 with ports to send / receive UDP messages open. Bluetooth, I2C, I2S functionality must be enabled.
- Some pi's may require adjustments to the bluetooth settings to connect to the iCreate3 Bluetooth. See links for the specific build of Ubuntu 22.04 if you have issues.
- See the detailed set-up guide for additional guidance on configuring a Pi for use with the ICreate 3, the PIBOT script and ROS2. It documents our configuration of an Ubuntu 22.04.03 image. Your hardware and specific Ubuntu or Raspbian build may require adjustments.
- Bluettoth pairing: On occasion some Pi devices experienced bluetooth pairing and connection issues to the iCreate3. There are a wide range of guides to troubleshoot bluetooth connection and pairign issues. We have found that delaying the start of the bluetooth service may help, as does cycling bluetooth on/off from the command line.
The hub/sequencer allows for control of multiple robots and multiple pi's from a unified OSC interface.
- You must configure the iCreate ROS namespace and device name to have unique names for each bot in the system.
- See the Network Diagram for single bot or the examples page for more detailed instructions.
- you may need to adjust the UFW or other firewall settings of your local router and the individual pi's to allow for peer to peer UDP transmission.
Please see the EXAMPLES page in the docs folder for example files, downloads and additional configuration resources.
The Pibot script will expect to find the following environment variables:
MY_IP- the script host's local IP addressMY_ID- the assigned bot's ID, an integer numberMY_ROBOT_NAME- the assigned robot's Bluetooth device namePIHUB_OSC_IP&PIHUB_OSC_PORT- IP address and port for OSC communications to the PihubISADORA_OSC_IP&ISADORA_OSC_PORT- IP address and port for OSC communications to the Pihub
Note: If there are no hub clients (i.e. Pihub & Isadora) then the environment variables can be placeholder IPs and ports.
Environment variables should be set up by editing and then running the scripts/env.sh script, or passed in directly at the shell.
Once installation and configuration is complete, from the Pi command line enter
cd OSC_PIBOT/scripts
source ./env.sh
source ./pibot-start
You should begin seeing status messages display on the pi terminal window.
To confirm communication try sending a OSC message such as /pibot/command/play-note from an OSC source. The robot should respond, and you should also see status messages display confirming. You can also watch the incoming OSC address & port you set for the update streams below such as /pibot/update/battery.
On the PIHUB Pi,
cd OSC_PIBOT/scripts
source ./env.sh
source ./pihub-start
The following are a list of OSC messages in use along with their arguments and targets. Messages may be generated or received by any OSC source. Examples.md has links to example file implementing these messages for a variety of software platforms, and pibot_osc.py implements handlers for the messages.
/pibot/update/battery-> int (bot ID), int (0 to 100)- sender(s): Pibots
- receiver(s): Hub, Isadora
/pibot/update/position-> int (bot ID), float (x), float (y) - (can be normalised)- sender(s): Pibots
- receiver(s): Hub, Isadora
/pibot/update/is-docked-> int (bot ID), bool- sender(s): Pibots
- receiver(s): Hub, Isadora
/pibot/update/going-to-charge-> int (bot ID)- sender(s): Pibots
- receiver(s): Hub, Isadora
/pibot/update/charging-> int (bot ID)- sender(s): Pibots
- receiver(s): Hub, Isadora
/pibot/update/done-charging-> int (bot ID)- sender(s): Pibots
- receiver(s): Hub, Isadora
/pibot/command/stop-> bool- sender(s): Isadora
- receiver(s): Hub (-> Pibots)
/pibot/command/play-note-> int- sender(s): Max Patch, Isadora
- receiver(s): Hub (-> Pibots)
/pibot/command/go-to-point-> int (robot ID), int- sender(s): Isadora
- receiver(s): Hub (-> Pibots)
/pibot/command/set-LED-> rgbw (0..255) (Pi #, R, G, B, W, R2, G2, B2, W2) (expandable to 16 channels)- sender(s): Isadora
- receiver(s): Hub (-> Pibots)
/pibot/command/undock-> (port, IP, Robot ID)- sender(s): Isadora / Max
- Receiver: Pibots
/pibot/command/dock-> (port, IP, Robot ID)- sender(s): Isadora / Max
- Receiver: Pibots
/pibot/command/rotate-> (port, IP, degrees, degrees per second, robot ID)- sender(s): Isadora / Max
- Receiver: Pibots
/pibot/command/move-> (port, IP, robot ID, Distance, Speed)- sender(s): Isadora / Max
- Receiver: Pibots
/pibot/command/navigate-to-> (port, IP, robot ID, x location, y location, Speed)- sender(s): Isadora / Max
- Receiver: Pibots
/pibot/command/callibrate-rotation-> (port, IP, robot ID)- sender(s): Isadora / Max
- Receiver: Pibots
/pihub/reload-cues-> (hub port, hub IP)- sender(s): Isadora / Max
- Receiver: Pibots
/pihub/set-scene-> (hub port, hub IP, scene #)- sender(s): Isadora / Max
- Receiver: Pibots
The OSC PIBOT framework was originally developed for the Dream Club Lab,** a light and media art installation by Ian Winters and Elaine Buckholtz, as a Nighthouse Studios collaboration, with a music score by [Evelyn Ficarra](https://evelynficarra.net, and robot programming Dimitris Kyriakoudis. Dream Club Lab was commissioned by the San Jose Downtown Association through the support of the City of San Jose and the Knight Foundation, with production support by 37 North. Learn more about the Dream Club Lab project.
Coding for OSC_PIBOT framework by Dimitris Kyriakoudis with Ian Winters. Hardware and framework concept by Ian Winters.
Further development of the OSC PIBOT framework takes place through the AI Perform Initiatve at the University of Sussex led by Professor Evelyn Ficarra, through 37 North and the Milkbar. The Dream Club Lab will also be a course at Mass College of Art by Professor Elaine Buckholtz.
Additional contributions to the development of artist oriented robotics software are welcomed as well as workshop inquiries.
The OSB PIBOT framework is released under the BSD 3-Clause License with no warranty of any kind. It uses the following open source direct dependencies, with license terms incorporated by reference, as well as those of any sub-dependencies. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS “AS IS” AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
- iRobot-edu-python sdk licensed uder the BSD 3-Clause License, available here
- Bleak, licensed under the MIT license, available here
- pySerial, licensed under the BSD 3-Clause license, available here
- Root Robot Python Web App, licensed under the MIT license, available here
- python-osc, Public Domain
- Python 3, under the Python Software Foundation license
- Adafruit blinka and adafruit pca9685 libraries licensed under the MIT license, available here
- rclpy licensed under the Apache 2 license availble here
- ROS2 libraries from the BSD 3-Clause license
- Clojure, licensed under an EPL 1.0 license
- Leiningen, licensed under an EPL 1.0 license