openFramworks addon for OpenAI's Whisper using whisper.cpp
Developed with openFramworks v0.12.0 Should work with newer versions as well as v0.11.2
install this addon in the default addons folder of your openFramworks install.
cd your_openframeworks_folder/addons/
git clone https://github.com/roymacdonald/ofxWhisper.git
This addon requires you to also have installed ofxSampleRate. Please check its readme file to get it installed properly.
This addon already includes the library compiled for Macos so you dont need to build it again. It works on both, Intel and Apple CPUs.
For Apple processors (M1 and M2) it was built with support for CoreML which gives it a very noticeable processing speed increase. For this you will need to download additionaly the coreML model using the script in libs/whisper_cpp/models/download-coreml-model.sh
Run the build_lib.sh
script to download and compile for your platform.
This will download also the basic model. Look at whisper.cpp' readme on how to download other models.
If you are not using macos make sure that before runinng the script you modify the line that reads
cp libwhisper.a ../libs/whisper_cpp/lib/osx/
and change osx
so it matches your platform.
cd your_openframeworks_folder/addons/ofxWhisper
.build_lib.sh
It is very important that you download the necesary models.
For Linux/Macos use the following script
libs/whisper_cpp/models/download-ggml-model.sh
for windows use:
libs/whisper_cpp/models/download-ggml-model.cmd
Either of these scripts need to be called with the model name as argument. like
libs/whisper_cpp/models/download-ggml-model.sh base.en
which will download the base model only for english language.
For more info check here
For these you will need to download additionaly the coreML model using the script in libs/whisper_cpp/models/ download-coreml-model.sh
. You need to pass as argument the same you did for the regular model.
For example:
libs/whisper_cpp/models/download-coreml-model.sh base.en
.
There are 3 examples.
- example_simple
- example_ofxSoundObjects_AudioInput
- example_ofxSoundObjects_AudioPlayer
For the last two you will need ofxSoundObjects and ofxAudioFile installed.
This will open a sound input (microphone) on your computer and process it with whisper. The results will be drawn on screen. Make sure you choose the correct sound device.
Uses ofxSoundObjects. The way it works is the same as example_simple but it connects trhough a chain of ofxSoundObjects. This allows you to either process the sound input before or after passing trhough whisper, using ofxSoundObjects. Check ofxSoundObjects examples to see how to work with it.
Uses ofxSoundObjects. Loads an audiofile, plays it and proceeses it with whisper.
Remember to update the examples project using Project Generator