Saxelerophone

Saxophonists can move their instrument around in space relatively easily when playing. In fact, it is quite common to see this kind of demonstration in live performances. Although these musical gestures demonstrate a certain virtuosity on the part of the performer, they are not directly linked to sound production.

Based on two important concepts of the field of Interactive Music System (IMS): the spare bandwidth adn the demonstration of virtuosity, we propose the Saxelerophone: a hyper-instrument tracking data from a 3-axis digital accelerometer and a contact microphone to create new interactive sounds for the saxophone.

Design and implementation

Based on the initial research work carried out on the saxophone and its musical gestures, we defined three main objectives for demonstrating gestural virtuosity using the Saxelerophone.

Audio sensing for providing information of the notes played on the instrument.
Gesture sensing for the prototyping of mappings strategies using machine learning.
Motion sensing for demonstrating gestural virtuosity.

Sensing objectives are achieved through the use of a contact microphone and an ADXL337 3-axis digital accelerometer, positioned one on top of the other on the upper part of the saxophone’s mouthpiece. Sensing data is processed using the visual programming language Pure Data (version 0.52.1) and runs on a Bela board.

A functional diagram of the Saxelerophone with its electronic components and how they are connected is presented in Figure 2.


Figure 2: Functional diagram of the Saxelerophone.

Gesture-sound mapping

Mapping of the gestures into sound synthesis parameters is performed using machine learning. A schematic representation of the Saxelerophone’s mappings is presented in the diagram in Figure 3.


Figure 3: Saxelerophone’s mappings.

A standard regression algorithm is trained to build a new creative sound space for the performer. The algorithm is optimized using the Adam optimizer initialized with a default learning rate of 0.001 and a weight decay of 0.0001. Training of the algorithm is facilitated using the ANN framework for Pure Data called neuralnet.

Installation

Software

Install the collection of Pure Data abstractions SFApdlib, and copy the folder sfapdlib and its contents in the computer and the Bela Pure Data externals directory.
Install the pd-externals [neuralnet] on Bela (external already compiled) and on your computer (compile it or download it).
- Bela: download this neuralnet folder, unzip it and move it to /root/Bela/projects/pd-externals/neuralnet (e.g. use Filezilla).
- Computer:
  1. Compile the pd-externals [neuralnet] on your computer (optional)
    
    Ensure you have the GNU make (version >= 3.81 required) and download or clone the neuralnet repository.
    - Go to the neuralnet folder and run make on your terminal
      - if make fails it might be because it can't find m_pd.h. You can download the file from here and put it in the neuralnet folder.
      - if make succeeds you will find two new files in the neuralnet folder called neuralnet.pd_xxx and neuralnet.pd_xxx.o, where xxx is a string corresponding to your operating system.
      - Open neuralnet-help.pd and ensure that all objects load (i.e. no red error message in the PD window).
  2. Download the pd-externals [neuralnet] on your computer
  - Intel-based mac: download this neuralnet folder, unzip it and move it to your pd-external folder.
  - Windows: download this neuralnet folder, unzip it and move it to your pd-external folder.
  - Open neuralnet-help.pd and ensure that all objects load (i.e. no red error message in the PD window).
  - Create a sub-folder called neuralnet in your pd-externals folder, and move all the files inside.

Hardware

Solder the wires shown in Figure 2 between the sensors and the Bela board. Use wires with a minimum length of 50 cm and a blank Printed Circuit Board (PCB) place on top of the Bela board. Hang the Bela board around your neck with a strap and adjust it so that the board is roughly in the middle of your torso. In a second step, place the contact microphone on the upper part of the saxophone ligature, with the ADXL337 3-axis digital accelerometer placed on top of it. Secure the electronic components with electrical tape and perform a continuity test using a multimeter to see if current flows properly. Figure 4 shows the electronic components involved and their location on the Saxelerophone mouthpiece.


Figure 4: Location of the Saxelerophone's electronic components.

Usage

Training the regression model

Connect your Bela system to your computer with a USB cable. Open the training folder of this repository, move the _main.pd patch to your project on the Bela IDE, and train a regression model with the train.pd patch running on your computer. This enables to create new regression mappings for the Saxelerophone.

Play with the Saxelerophone 🎷!

Connect your Bela system to your computer with a USB cable. Move the _main.pd and reg_model.ann of this repository to your project on the Bela IDE, go to the settings and set the block size to 512. Finally, set Run on Boot and play with the Saxelerophone!

Contributing

Pull requests are welcome. For major changes, please open an issue first to discuss what you would like to change. Please make sure to update tests as appropriate.

Acknowledgements

We would like to thank Alexandros Drymonitis from neuralnet for open-sourcing his code and publicly releasing checkpoints, and Stefano Fasciani for the useful collection of Pure Data abstractions SFApdlib.

joachimpoutaraud/saxelerophone