/whale-speech

A pipeline to map whale sightings to hydrophone audio

Primary LanguagePythonGNU General Public License v3.0GPL-3.0

📣🐋 Whale Speech

A pipeline to map whale encounters to hydrophone audio.

Derived from PacificSoundDetectHumpbackSong, though not directly affiliated with MBARI, NOAA, or HappyWhale.

Installation

M1:

CONDA_SUBDIR=osx-arm64 conda create -n whale-speech python=3.11
conda activate whale-speech
pip install -r requirements.txt

Other:

conda create -n whale-speech python=3.11
conda activate whale-speech
pip install -r requirements.txt

Pipeline description

Stages:

  1. Input: When (and where*) to look for whale encounters on HappyWhale.

  2. Geometry Search: Query open-oceans/happywhale to find potential whale encounters.

    → Expected outputs: encounter ids, start and end times, and longitude and latitude.

  3. Retrive Audio: Download audio from MBARI's Pacific Ocean Sound Recordings around the time of the encounter.

    → Expected outputs: audio array, start and end times, and encounter ids.

  4. Filter Frequency: Break audio into non-overlaping segments with flagged frequency detections.

    → Expected outputs: cut audio array, detection intervals, and encounter ids.

  5. Classify Audio: Use a NOAA and Google's humpback_whale model to classify the flagged segments.

    → Expected outputs: resampled audio, classification score array, and encounter ids.

  6. Postprocess Labels: Build clip-intervals for each encounter for playback snippets.

    → Expected outputs: encounter ids, cut/resampled audio array, and aggregated classification score.

  7. Output: Map the whale encounter ids to the playback snippets.

*Currently only support encounters around the Monterey Bay Hydrophone (MARS).

Resources