MMW Signifier
"Signifier" is comprehensive platform to run the interactive sound & light sculpture on Raspberry Pi 4. Each of the 15 Signifiers allow user interactive through integrated Bluetooth scanning, and microphone input analysis, which shapes the interactive audio composition and LED reactivity. Created by Chris Vik in collaboration with office.org.au for Melbourne Music Week 2021/2022.
|
| A demo setup of the physical Signifier units. |
Each of the 15 Signifiers use a Raspberry Pi 4B (4GB) fitted with several hardware devices to provide interaction with the physical world.
This application manages a suite sensor input modules, sources, and output modules, destinations. A Signifier's behaviour is customisable via a scheduler and value mapping system, which allow total control over the automation and interaction between inputs and outputs, even during run-time. The project includes a management web application running on each Signifier, and all the Signifier data is recorded on a local Prometheus database, which can be viewed in real-time using Grafana.
|
| The Signifier management web-app, "Sig-Config". |
|
| Visualising Signifier data in real-time with Grafana. |
Contents
Project features
Codebase
- Full multi-processor support.
- Automation scheduling manager.
- Source/destination value mapping system.
- Dynamic audio composition playback system.
- Composition output stream audio analysis.
Hardware integration
- Duplex serial communication with Arduino for real-time LED control.
- Anonymised Bluetooth (BLE) activity scanner metrics.
- Microphone input analysis. (not yet implemented)
- Temperature sensor input. (not yet implemented)
Configuration and management
- Local configuration serialisation.
- Integrated configuration and event API. (not yet implemented)
- Integrated web-app for convenient config/event API access and basic monitoring. (not yet implemented)
- Docker/Portainer agent for container management from the Signifier Server.
Metrics and networking
- Local Prometheus push gateway for complete metrics capture.
- Custom VPN networking for secure remote access, during both development and production.
- Prometheus federation server for metrics aggregation.
- Grafana dashboards for real-time monitoring of all metrics.
Hardware
Note: Hardware fit-out was specified and purchased prior to my involvement. I have added notes on sticking-points, work-arounds and incompatibilities with provided hardware.
- Raspberry Pi 4B (4GB) - link
- Arduino Nano Every - link
- 4m x RGB LED strips (WS2812B 60/m) - link
- Audio amplifier (50 watt Bluetooth) - link
- Digital Temp Sensor - link
- Mini USB Microphone - link
Software
Core
- Raspberry Pi OS - link - (Debian Bullseye) 64-bit/arm64
- Python 3.9
System modules
- libasound2-dev - link - Downloads ALSA C-libraries for Python library compilation. (might not be needed, will test without on a new build)
- alsa-utils - link - Provides loopback and additional debugging tools.
Python modules
- schedule - link - Required for scheduling "jobs" to automate the Signifier.
- pygame - link - Back-end framework for audio clip playback.
- pyAlsaAudio - link Wrapper for ALSA, required for analysis module.
- PySerialTransfer - link - Arduino communication framework, required for LED module.
- prometheus-client - link - Required if using Prometheus/Grafana to monitor Signifiers.
Docker & containers
- Docker - link - Container framework
- Docker Compose - link - More convenient method of container management.
- Portainer Agent - link - Remote management of Docker environment.
- Prometheus - link - Local time-series database for recording system metrics and sensor data.
Networking
By default, Signifiers will look for the WiFi SSID mmw_sig_net, and use the password redacted
Once connected, they will automatically attempt connection via VPN. The VPN provides a secure network for the Signifiers to communicate with the Signifier Server, which records metrics, allows for remote SSH access to the Signifiers, and hosts several front-end web-apps for remote monitoring and configuration of the Signifiers.
See the Signifier Access section for more details.
Audio content
Location
All media content for the Signifiers should go into the Signifier application's /media subdirectory (default /home/pi/Signifier/media). There is a subdirectory called audio which contains the Signifier's audio clip collections.
The default path for audio resources supplied on delivery is here:
/home/pi/Signifier/media/audio/
There are 11 separate "collections" of Signifier audio presets: S1, S2... S11.
Format
The expected format for audio content used by the Signifier application is:
- File extension:
.wav - Sample rate:
48000 - Channels:
1/mono - Bit depth:
16-bit/signed 2-byte integer
While the format of audio clips can be converted in real-time by the application, it uses a large amount of CPU to do so. Supplying the incorrect format will heavily impact performance, and may result in interrupted audio playback and incorrect audio analysis outputs.
Replacing / adding audio collections
I suggest using the free Windows application FFMpeg Batch AV Converter to convert large quantities of audio files to the desired format. The following conversion command will produce the correct output:
-vn -c:a pcm_s16le -sample_fmt s16 -ar 48000 -ac 1
Once converted, simply add the folder containing the new collections into the media/audio directory, and ensure the config.json collection base_path setting points to the new audio library path.
Installation
Option 1: SD card duplication
Use an application like BalenaEtcher to write the supplied Signifier image on to a fresh SD card. This is by far the easiest and quickest method to deploy a new Signifier.
Note: Each Signifier should have a unique hostname that reflects the assigned letter of its physical component. When duplicating the Signifier image, the hostname should be changed from within the operating system. This is especially important to do if the Signifiers are intended to be deployed in a networked or VPN environment.
Option 2: Install script
Should the SD card duplication not work, or if you'd like to build the Signifier environment on a fresh environment, you can use the setup.sh script supplied in this repository.
On any Windows, OS X, or Linux system:
-
Download the the Raspberry Pi Imager software: https://www.raspberrypi.com/software/
-
Open the imager software, select Raspberry Pi OS (other) > Raspberry Pi OS Lite (64-bit)
-
Chose the SD card from the "Choose Storage" option.
-
Click the gear icon, and complete the following details:
-
Hostname:
- Signifier name format:
mmwSig<letter>- Where
<letter>is the capital letter of the Signifier being deployed (e.g. "A", "B"... "N"): - For example:
mmwSigF
- Where
- Signifier name format:
-
Enable SSH:
- Use password authentication
-
Set username and password
- Username:
pi - Password =
redacted
- Username:
-
Configure wifi:
- SSID =
mmw_sig_net - Password =
redacted - WiFi country =
AU
- SSID =
-
Set locale settings
- Time zone =
Australia/Victoria - Keyboard layout =
us - Skip first-run wizard =
checked
- Time zone =
-
-
Hit
SAVEthenWRITE, and wait for the image to finish installing and verifying. -
Before removing the SD card from the reader, open the newly created drive called
BOOTfrom your file explorer. -
Copy and paste the
sig-contentsdirectory provided into the root of the SD card'sBOOTdrive. -
Remove the SD card from the computer and insert into the RPi, the connect the USB-C power cable.
Now it's time to install the Signifier application on your fresh Raspberry Pi OS:
-
Once the Pi has booted, login using the credentials your provided during installation, with username
pi. -
Install the
gitmodule:sudo apt install git
-
Clone this repo:
git clone https://github.com/MaxVRAM/Signifier.git && cd Signifier
-
Execute the install script and follow any prompts:
./setup.sh
- NOTE: The most straight-forward and comprehensive installation option is to enter yes to the first two options. This will install and copy/download all content and configure the system for all Signifier functionality. To customise the setup and select individual elements, enter no for the second option.
-
After installation, reboot the Raspberry Pi to start the Signifier.
NOTE: It's possible that the Signifier monitoring apps could not be installed during the setup.sh procedure. These can be installed by running the ./update-monitoring.sh command from within the Signifier directory on the Pi.
Signifier Access
There are several methods for accessing a Signifier. The method you use will depends on 2 things:
- What you need to accomplish.
- What kind of access you have to the Signifier.
There are 3 primary methods of accesss:
Method 1: Browser accesss
If you're on the same WiFi network (or VPN) as the Signifier, you can access the following web applications hosted on the Signifier via your browser:
Sig-Config
Update the Signifier configuration.
URL: <signifier-ip-address>:5000
Functionality:
- Download the current Signifier configuration files.
- Upload new or modified configuration files.
- Restart the Signifier.
- Most Signifier modules will apply updated configurations during runtime without requiring a restart.
Grafana
View Signifier data graphed on tidy dashboards.
URL: <signifier-ip-address>:3000
Functionality:
- Monitor the status of online Signifiers.
- View pretty graphs displaying sensor, composition, LED, and system data.
- Useful when updating Signifier configuration.
Prometheus/Metrics Push Gateway
Provides the sensor data to Grafana.
URL: <signifier-ip-address>:9091
Functionality:
- View latest Signifier data if Grafana is inaccessable.
- Provides an API point for custom applications to ingest Signifier data.
Method 2: SSH
SSH provides a command line interface (CLI) into the Signifier operating systems (OS). This will provide essentially the same functionality as having a mouse and keyboard connected to the Signifier Raspberry Pi from a remote computer, but requires network access to the Signifier.
On OS X platforms, you can follow the commands below from the Terminal app. On Windows, this is available from either Command Line, or PowerShell.
Once in a terminal application on your computer, you can access a Signifier remotely via SSH with the following command:
ssh pi@<signifier-ip-address>You'll then have to enter the password for the pi user of the Signifier (that you provided in the installation steps).
After which, you'll be connected to the Signifier OS CLI.
Signifier Scripts
-
Navigate to the Signifier directory:
cd ~/Signifier
-
Once in the Signifier directory, you can run the Signifier setup/installation script:
./setup.sh
-
Perform a full update and refresh, restoring default Signifier configurations:
./refresh.sh
-
Update the Signifier code (if it's been updated on GitHub):
./update-app.sh
-
Re-compile and upload the LED code to its connected Arduino:
./update-arduino.sh
-
Restart the Signifier monitoring Docker containers (Grafana and Prometheus):
./update-docker.sh
Signifier Services
From anywhere in the OS, you can check the status of the Signifier system services. These services are configured during the initial Signifier setup.sh process, and try to ensure critical Signifier applications are kept running. Should there be a critical system fault, the service may not be able to maintain the online status of a Signifier application.
-
You can check the status of a Signifier service via the following commands:
sudo systemctl status signifier- status of the Signifier application responsible for the primary Signifier functionality.sudo systemctl status sig-config- status of the Sig-Config web-application.sudo systemctl status openvpn@client.service- status of the Signifier's access to the Sig-Net VPN.
-
If a service returns a
disabledstatus, it can be re-enabled using the same command, only replacingstatuswithenable. For example:sudo systemctl enable signifier -
When the system is rebooted, the service should automatically start. The service can be started immediately using the
startkeyword. For example:sudo systemctl start signifier
General OS Commands
-
Reboot the Signifier Raspberry Pi:
sudo reboot
-
Shutdown the Signifier Raspberry Pi:
sudo poweroff
-
Test the audio output:
speaker-test -t wav # Use CTRL-C to abort the test at any time.
Method 3: Physical access
If you are able to connect a monitor and keyboard (via micro HDMI and USB ports, respectively), you are afforded the same functionality as SSH (explained above), only without the requirement of network access.
This should only be required if the Signifier is inaccessible via SSH, in instances where the Signifier is unable to find a valid WiFi connection, or where the OS is unable to start critical system functionality (for example, if the SD card has been corrupted).
See Method 2 for examples of functionality in this mode.