react-audio-spectrogram-player

An audio player written in React computes and shows a mel scale spectrogram along with the audio. The playhead on the spectrogram is synchronized with the audio player. You can zoom in on the spectrogram and slow down the audio. It acts almost as a microscope for audio.

The spectrogram is computed using web assembly. The Rust code for this can be found here.

If you use Python, there is also a jupyter widget based on this component. You can find it here.

Preview

See the demo.

Installation

npm i react-audio-spectrogram-player

Usage

Basic Usage

In a React component:

import SpectrogramPlayer from "react-audio-spectrogram-player";
import src from "./path/to/your/audio/file.wav";

const App = () => {
  return <SpectrogramPlayer src={src} />;
};

Customization

To override the spectrogram values (suppose you precomputed the spectrogram values), you can use the sxx prop.

return (
  <SpectrogramPlayer
    src={src}
    sxx={sxx} // number[][]
  />
);

There are several other props to customize the spectrogram and the interface.

return (
  <SpectrogramPlayer
    src={src}
    sampleRate={16000}
    n_fft={1024}
    win_length={400}
    hop_length={160}
    f_min={20}
    f_max={8000.0}
    n_mels={128}
    top_db={80}
    annotations={annotations}
    startTimeInitial={2.0}
    endTimeInitial={4.0}
    playbackSpeedInitial={0.5}
    playheadModeInitial="loop"
    navigator
    settings
    specHeight={300}
    navHeight={80}
    colormap="inferno"
    transparent
    dark
  />
);

Prop	Type	Default	Description
`src`	`string`	required	Path to the wav audio file.
`sxx`	`number[][]`	undefined	2D array with spectrogram values. It will override the spectrogram computed from the audio.
`sampleRate`	`number`	`16000`	Sample rate used when loading audio.
`n_fft`	`number`	`1024`	Number of FFT bins.
`win_length`	`number`	`400`	STFT Window length.
`hop_length`	`number`	`160`	STFT Hop length.
`f_min`	`number`	`0`	Minimum frequency corresponding to the first mel band.
`f_max`	`number`	`sampleRate/2`	Maximum frequency corresponding to the last mel band.
`n_mels`	`number`	`128`	Number of mel bands.
`top_db`	`number`	`80`	Maximum decibel value.
`annotations`	`Annotations[]`	`[]`	List of annotations to display (see Annotations section).
`navigator`	`boolean`	`false`	Allow user to zoom in with the navigator UI.
`settings`	`boolean`	`false`	Allow user to change some playback behaviour.
`startTimeInitial`	`number`	`undefined`	Zoom using this start time on load.
`endTimeInitial`	`number`	`undefined`	Zoom using this end time on load.
`playbackSpeedInitial`	`number`	`1.0`	Initial playback speed.
`playheadModeInitial`	`string`	`'page'`	Initial playhead mode.
`specHeight`	`number`	`200`	Height of the main spectrogram.
`navHeight`	`number`	`50`	Height of the navigator UI.
`colormap`	`string`	`'viridis'`	The colormap to use.
`transparent`	`boolean`	`false`	Use rgba values for spectrogram image.
`dark`	`boolean`	`false`	Use dark mode theme.

Annotations

You can annotate intervals (such as words or phones) below the spectrogram.

First place your annotation data in one or more (string | number)[][] objects.

Column 1: Start time in seconds
Column 2: End time in seconds
Column 3: Annotation as a string

For example:

const wordIntervals = [
    [0.54, 0.84, "this"],
    [0.84,  1.1, "little"],
    [ 1.1,  1.4, "work"],
    ...
]
const phoneIntervals = [
    [0.54, 0.62, "ð"],
    [0.62, 0.67, "ɪ"],
    [0.67, 0.84, "s"],
    ...
]

Now create a list of annotation objects that will be passed to the component:

const annotations = [
  {
    data: wordIntervals,
    title: "Word intervals:",
    height: 30,
    strokeWidth: 1,
  },
  {
    data: phoneIntervals,
    title: "Phone intervals:",
    height: 30,
    strokeWidth: 1,
  },
];

The title, height and strokewidth entries are optional.

Pass the list to the component via the annotations prop:

return (
  <SpectrogramPlayer
    src={src}
    sxx={sxx}
    specHeight={200}
    navHeight={50}
    annotations={annotations}
  />
);

Future Updates

Better touch screen compatibility.

Japh/react-audio-spectrogram-player